Content set based deltacasting

ABSTRACT

Methods, apparatuses, and systems are provided for improving utilization of the satellite communications system through various “deltacasting” techniques for handling content sets (e.g., feeds or websites). Embodiments operate in a client-server context, including a server optimizer, a client optimizer, and, in some embodiments, a pre-positioning client. Within this client-server context, content sets are multicast (e.g., anticipatorily pre-positioned in a local dictionary) to end users of the communications system and are handled at the content set level, according to set-level metadata and/or user preferences. In some embodiments, when locally stored information from the content sets is requested by a user, deltacasting techniques are used to generate fingerprints for use in identifying and exploit multicasting and/or other opportunities for increased utilization of links of the communications system.

CROSS-REFERENCES

This application claims the benefit and is a continuation of U.S.application Ser. No. 13/919,888, filed on Jun. 17, 2013, entitled“CONTENT SET BASED DELTACASTING”, which is a divisional of U.S.application Ser. No. 12/685,729, filed on Jan. 12, 2010, titled,“CONTENT SET BASED DELTACASTING,” which is a non-provisional of U.S.Provisional Application Ser. No. 61/144,363, filed on Jan. 13, 2009,titled “SATELLITE MULTICASTING,” and U.S. Provisional Application Ser.No. 61/170,359, filed on Apr. 17, 2009, titled “DISTRIBUTED BASE STATIONSATELLITE TOPOLOGY,” all of which are hereby expressly incorporated byreference in their entirety for all purposes.

This application is also related to U.S. patent application Ser. No.12/651,909, titled “DELTACASTING,” filed on Jan. 4, 2010, which ishereby expressly incorporated by reference in its entirety for allpurposes.

BACKGROUND

This disclosure relates in general to communications and, but not by wayof limitation, to multicast optimization over links of a communicationssystem.

In some topologies of communications systems, groups of users share someor all of the forward link. For example, in some satellitecommunications systems, users share spot beams for communicating with aservice provider (e.g., via a base station and/or gateway).Communication services provided to the users over the shared forwardlink may be affected by a number of factors, including bandwidth andother link conditions. For example, because all users sharing theforward link also share the link's bandwidth, any unnecessaryredundancies in communications may cause sub-optimal utilization of theforward link.

As such, it may be desirable to optimize utilization of the sharedforward link by minimizing redundancies.

SUMMARY

Among other things, methods, systems, devices, and software are providedfor improving utilization of the satellite communications system throughvarious “deltacasting” techniques for handling content sets (e.g., feedsor websites). Embodiments operate in a client-server context, includinga server optimizer, a client optimizer, and, in some embodiments, apre-positioning client. Within this client-server context, content setsare multicast to end users of the communications system and are handledat the content set level, according to set-level metadata and/or userpreferences. In some embodiments, when locally stored information fromthe content sets is requested by a user, deltacasting techniques areused to generate fingerprints for use in identifying and exploitingmulticasting and/or other opportunities for increased utilization oflinks of the communications system.

In one set of embodiments, a method is provided for multicasting over acommunications system having a communications path between a server sideof the communications system and a plurality of clients, thecommunications path including a shared forward link over which bandwidthresources are shared during a multicast communication. The methodincludes: intercepting a data block at the server side of thecommunications system, the data block having a header portion and acontent portion and being communicated as part of a content set over thecommunications path, the content set having a plurality of contentobjects and being identified by a content set identifier; associatingthe data block with the content set identifier; generating a fingerprintusing byte-level information comprised by the content portion of thedata block; determining whether to multicast the data block over thecommunications path according to the fingerprint; and when it isdetermined to multicast the data block: multicasting the data block overthe communications path; and using the content set identifier todetermine whether a client accepts the data block multicast over thecommunications path. In certain embodiments, using the content setidentifier to determine whether the client accepts the data blockmulticast over the communications path includes determining, at theserver side of the communications system, whether to include the clientin a multicast group, wherein the data block is multicast over thecommunications path according to the multicast group. In otherembodiments, using the content set identifier to determine whether theclient accepts the data block multicast over the communications pathincludes determining, at the client, whether to locally store the datablock according to the content set identifier.

Further areas of applicability of the present disclosure will becomeapparent from the detailed description provided hereinafter. It shouldbe understood that the detailed description and specific examples, whileindicating various embodiments, are intended for purposes ofillustration only and are not intended to necessarily limit the scope ofthe disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is described in conjunction with the appendedfigures:

FIG. 1A shows a simplified block diagram of one embodiment of acommunications system for use with various embodiments;

FIG. 1B shows a simplified block diagram of another embodiment of acommunications system having multiple optimizer tunnels for use withvarious embodiments;

FIG. 2 shows a block diagram of an embodiment of a satellitecommunications system having a server system in communication withmultiple user systems via a satellite over multiple spot beams,according to various embodiments;

FIG. 3 shows a simplified block diagram illustrating an embodiment of aserver system coupled between a network and an antenna, according tovarious embodiments;

FIG. 4 shows a simplified block diagram of an embodiment of a usersystem, including an embodiment of a user terminal coupled between auser antenna and a CPE, according to various embodiments;

FIG. 5 shows a block diagram of an embodiment of a communicationssystem, illustrating client-server interactivity through a clientoptimizer and a server optimizer, according to various embodiments; and

FIG. 6 is a flow diagram of an illustrative method for usingdeltacasting to handle content set traffic over a communications system,according to various embodiments.

In the appended figures, similar components and/or features may have thesame reference label. Further, various components of the same type maybe distinguished by following the reference label by a dash and a secondlabel that distinguishes among the similar components. If only the firstreference label is used in the specification, the description isapplicable to any one of the similar components having the same firstreference label irrespective of the second reference label.

DETAILED DESCRIPTION

The ensuing description provides preferred exemplary embodiment(s) only,and is not intended to limit the scope, applicability, or configurationof the disclosure. Rather, the ensuing description of the preferredexemplary embodiment(s) will provide those skilled in the art with anenabling description for implementing a preferred exemplary embodiment.It is understood that various changes may be made in the function andarrangement of elements without departing from the spirit and scope asset forth in the appended claims.

Embodiments are described herein for improving utilization of thesatellite communications system through various “deltacasting”techniques for handling content sets. As used herein, “content sets”include any grouping of content intended to be called as a group andcharacterizable as a group by set-level metadata. In one example, acontent set is an item in a really simple syndication (RSS) feed,characterized by channel metadata and/or item metadata. In anotherexample, a content set is a web page, characterized by certain HTTPmetadata. When a user requests the content set (e.g., by entering auniform resource locator (URL), clicking on a link in an RSS reader,etc.), a set of content files is invoked.

For example, when a link to a “video” is selected in an RSS reader, thelink may, in fact, cause a web page to build using a number of calls toone or more content sources. The resulting web page may include therequested “video” and other content, including a banner advertisement, alogo image, etc. As such, the “video” link is actually a link to acontent set characterized at least partially by metadata associated withthe link at the set level (e.g., as opposed to the object-level metadataassociated with each content object file called as part of the contentset).

Various embodiments exploit content sets to provide set-level handlingfunctionality in the context of otherwise substantially transparentclient-server transactions. In effect, embodiments provide functionalityof a transparent, man-in-the-middle optimizer (e.g., a protocol-agnosticaccelerator) with a certain level of content awareness for additionalfunctionality. For example, content sets are multicast (e.g.,anticipatorily pre-positioned in a local dictionary) to end users of thecommunications system and are handled (e.g., at the client side) at thecontent set level, according to set-level metadata and/or userpreferences. In some embodiments, when locally stored information fromthe content sets is requested by a user, deltacasting techniques areused to generate fingerprints for use in identifying and exploitingmulticasting and/or other opportunities for increased utilization oflinks of the communications system.

Referring first to FIG. 1A, a simplified block diagram is shown of oneembodiment of a communications system 100 a for use with variousembodiments. The communications system 100 a facilitates communicationsbetween a user system 110 and a content server 150 via a clientoptimizer 120, a server optimizer 130, and a network 140. The clientoptimizer 120 and the server optimizer 130 are configured to effectivelyprovide an optimizer tunnel 105 between the user system 110 and thecontent server 150, including providing certain communicationsfunctionality.

Embodiments of the optimizer (e.g., the server optimizer 130, the clientoptimizer 120, and the resulting optimizer tunnel 105) can beimplemented in a number of ways without departing from the scope of theinvention. In some embodiments, the optimizer is implemented as a proxy,such that the server optimizer 130 is a proxy server, the clientoptimizer 120 is a proxy client, and the optimizer tunnel 105 is a proxytunnel. For example, a transparent intercept proxy can be used tointercept traffic in a way that is substantially transparent to users atthe client-side of the proxy tunnel. In other embodiments, the optimizeris implemented as an in-line optimizer. For example, the clientoptimizer 120 is implemented within a user terminal and the serveroptimizer 130 is implemented within a provider terminal (e.g., asatellite base station or gateway, a cable head-end, a digitalsubscriber line access multiplexer (DSLAM), etc.). Other configurationsare possible in other embodiments. For example, embodiments of theserver optimizer 130 are implemented in the Internet cloud (e.g., oncommercial network leased server space). Embodiments of the clientoptimizer 120 are implemented within a user's personal computer, withina user's modem, in a physically separate component at the customerpremises, etc.

It is worth noting that references herein to “intercepting” data shouldbe construed broadly to include any useful slowing, sampling,re-routing, and/or other techniques that allow processing of the data asrequired according to various embodiments. In some embodiments, trafficpasses through the server optimizer 130, where it is “intercepted” bybeing buffered for analysis and processing. For example, the bufferingmay be used to slow and accumulate traffic for fingerprint generationand analysis, as described more fully below. Notably, certainembodiments described as using an optimizer component (e.g., the serveroptimizer 130) to intercept the traffic may actually be implemented byhaving a different component intercept the traffic, from which theoptimizer component may receive the intercepted traffic for processing.

Embodiments of the user system 110 may include any component orcomponents for providing a user with network interactivity. For example,the user system 110 may include any type of computational device,network interface device, communications device, or other device forcommunicating data to and from the user. Typically, the communicationssystem 100 a facilitates communications between multiple user systems110 and a variety of content servers 150 over one or more networks 140(only one of each is shown in FIG. 1A for the sake of clarity). Thecontent servers 150 are in communication with the server optimizer 130via one or more networks 140. The network 140 may be any type of network140 and can include, for example, the Internet, an Internet protocol(“IP”) network, an intranet, a wide-area network (“WAN”), a local-areanetwork (“LAN”), a virtual private network (“VPN”), the Public SwitchedTelephone Network (“PSTN”), and/or any other type of network 140supporting data communication between devices described herein, indifferent embodiments. The network 140 may also include both wired andwireless connections, including optical links.

As used herein, “content servers” is intended broadly to include anysource of content in which the users may be interested. For example, acontent server 150 may provide website content, television content, filesharing, multimedia serving, voice-over-Internet-protocol (VoIP)handling, and/or any other useful content. It is worth noting that, insome embodiments, the content servers 150 are in direct communicationwith the server optimizer 130 (e.g., not through the network 140). Forexample, the server optimizer 130 may be located in a gateway thatincludes a content or application server. As such, discussions ofembodiments herein with respect to communications with content servers150 over the network 140 are intended only to be illustrative, andshould not be construed as limiting.

In some embodiments, when the user system 110 communicates with thecontent server 150, the server optimizer 130 intercepts thecommunications for one or more purposes. As described below, the serveroptimizer 130 may be part of a server system 220 that includescomponents for server-side communications (e.g., base stations,gateways, satellite modem termination systems (SMTSs), digitalsubscriber line access multiplexers (DSLAMs), etc., as described belowwith reference to FIG. 2). The server optimizer 130 may act as atransparent and/or intercepting proxy. For example, the client optimizer120 is in communication with the server optimizer 130 over aclient-server communication link 125, and the server optimizer 130 is incommunication with the content server 150 over a content network link135. The server optimizer 130 may act as a transparent man-in-the-middleto intercept the data as it passes between the client-servercommunication link 125 and the content network link 135. Some purposesof the interception may include filtering, caching, parsing, and/orotherwise processing the requests and responses. For example, when theuser system 110 requests a web object from a content server 150, theserver optimizer 130 may intercept and parse the request to implementprefetching and/or other types of functionality.

As described more fully below, embodiments of the server optimizer 130use various techniques (e.g., dictionary coding) to identifyredundancies between incoming data and data previously sent across thelinks of the communication system 100 a (e.g., the client-servercommunication link 125 and the content network link 135). In particular,various techniques (e.g. delta coding, wide dictionary coding, etc.) mayallow identification of redundancies in byte sequences traversing thelinks even when a large history is maintained. These techniques may beused to identify and exploit opportunities for multicasting to increaseutilization of the communications links. Use of these techniques toidentify and exploit these multicast opportunities is referred to hereinas “deltacasting.”

It will be appreciated that “delta coding,” “dictionary coding,”“dictionary,” “deltacasting,” and other similar terms and phrases areintended to be broadly construed to include use of any type ofdictionary-like structure for optimization. Embodiments of thedictionary include chunks of content data (e.g., implemented as deltadictionaries, wide dictionaries, byte caches, and/or other types ofdictionary structures). For example, when content data is stored in thedictionary, some or all of the blocks of data defining the content arestored in the dictionary in an unordered, but indexed way. As such,content may not be directly accessible from the dictionary; rather, theset of indexes may be needed to recreate the content from the set ofunordered blocks.

It is worth noting that data may be communicated over a communicationssystem 100 a using one or more protocols that define, among otherthings, the format for the datagrams (e.g., packets, frames, etc.). Eachdatagram may typically include a header portion and a content portion.As used herein, the term “header” is intended broadly to include anyportions of the datagram other than those used to communicate the actualcontent (e.g., file data), and is not intended to be limited to anyparticular datagram format. For example, an Internet protocol (IP)packet may include a header at the beginning of each packet, while othertypes of datagrams may provide header-types of information in other ways(e.g., using preambles, post-ambles, mid-ambles, spread-ambles,sub-frames, separate signaling or control data, etc.). These headerportions may include information, such as source address, destinationaddress, priority, packet length, coding information, modulationinformation, etc. Of course, those of skill in the art will appreciatethat similar categories of header-portion and content-portioninformation may be found within datagrams of other protocol formats(e.g., HTTP, FTP, etc.).

Much can be gleaned from the header portions of data. For example, theheader portion may include metadata or other information about thecontent portion that can be used to help characterize the contentportion of the data. In fact, this technique may be used by certaintypes of content delivery systems, like a video-on-demand (VOD) system.A VOD system may include an application running at a VOD content serverand/or at the end viewer's customer premises equipment (CPE) (e.g., on aset-top box) for parsing and translating proprietary metadata frompacket headers of user requests. Notably, while use of the metadata mayprovide relatively straightforward knowledge of the content beingrequested, using proprietary tags in this way may require having accessto (e.g., and running an application on) the content server.

For example, a parsed URL may look as follows:“http://www.VOD.com/movieplayer?70AX05nkd4868PR1D5g.” The illustrativeURL includes a string of characters generated as part of a proprietaryapplication function, and may be decoded by the VOD server applicationto identify information, including the particular download requested, anidentifier for the session, user or account data, shopping cart data,client playback capabilities, etc. As such, another request for the sameVOD movie, even from the same content server, may have different URLs(e.g., different request headers). While the VOD application server maybe able to understand the requests as being for the same movie (e.g.,the VOD applications server will understand which bytes specify thecontent), a transparent intercept proxy, like that of embodiments of theserver optimizer 130, may not be able to determine this from themetadata alone.

Embodiments of the server optimizer 130 generate fingerprints (e.g.,fingerprints, digests, signatures, hash functions, etc.) from thecontent portion of the data traversing the communication links. Theserver optimizer 130 intercepts and analyzes the byte-level data of thecontent portion in a way that is substantially transparent to the user.Embodiments of the fingerprints are generated so as to be useful inidentifying redundancies between the incoming intercepted data andpreviously processed data. For example, hashing functions are applied totraffic, after being intercepted by the server optimizer 130, for use asidentifiers (e.g., “weak” identifiers) that are at least strong enoughto identify candidate matches with blocks stored in a dictionary. Someembodiments of the fingerprints are generated so as to be useful furtheras strong identifiers for representing substantially identical matchingblocks stored in a dictionary.

A number of difficulties arise from implementing this type of optimizerto use fingerprints (e.g., rather than metadata or other headerinformation). In one example, as described above, header data (e.g.,particularly proprietary metadata) may be used to make a number ofdeterminations (e.g., precisely what object file is being requested)that may be difficult or impossible to make from the content data alone.In another example, proprietary data or limited content environments mayallow certain assumptions to be made. For example, when someone requestsa VOD movie, the server may know exactly what bytes are being requested(e.g., whatever bytes are associated with that particular movie file onthe VOD server), how large the file is, that the viewer is likely towatch the movie sequentially, where the movie is stored, etc. However,by using the content portion of the data to generate fingerprints,embodiments of the server optimizer 130 are relatively agnostic to thecontent being analyzed, which may provide certain functionality evenwhere the server optimizer 130 has little or no access to proprietarymetadata and/or other header information.

In some embodiments, for example, the server optimizer 130 generatesfingerprints of data being received over the content network link 135 inresponse to various requests from different users on a shared spot beamof a satellite communications system (e.g., where the requests arefulfilled by the server optimizer 130 over the client-server link 125 ofthe communications system 100 a). The server optimizer 130 determinesfrom the fingerprints that multiple users are requesting the samecontent at substantially the same time. In response, the serveroptimizer 130 creates a multicast service flow (e.g., on theclient-server link 125) over which it multicasts the requested data toall the requesting users, thereby saving bandwidth relative tounicasting multiple copies of the content to the multiple users.

It is worth noting that embodiments of the client-server communicationlink 125 (e.g., between the client optimizer 120 and the serveroptimizer 130) and the content network link 135 (e.g., between theserver optimizer 130 and the content servers 150 via the networks 140)can be implemented as various types of links have different and/orchanging link characteristics, including, for example, differences inbandwidth, latency, cost per bit, etc. For example, while certainembodiments are described in the context of a satellite communicationssystem, where the client-server communication link 125 includes at leastone satellite link, other topologies and link types are possible.

While the communications system 100 a illustrated in FIG. 1A shows onlyone optimizer tunnel 105 between one server system 220 and one usersystem 110, embodiments typically operate in the context of, and takeadvantage of, multiple optimizer tunnels 105. FIG. 1B shows a simplifiedblock diagram of another embodiment of a communications system 100 bhaving multiple optimizer tunnels 105 for use with various embodiments.The communications system 100 b facilitates communications between aserver system 220 and multiple user systems 110, via a respective serveroptimizer 130 and multiple client optimizers 120. The client optimizers120 and the server optimizer 130 are configured to effectively providetunnels 105 between the user systems 110 and content servers 150.

A client-server communication link 125 between the server optimizer 130and the client optimizers 120 supports one or more unicast service flows525 and one or more multicast service flows 515 for supporting unicastand multicast traffic, respectively. In one embodiment, theclient-server communication link 125 includes a satellite communicationslink. It will be appreciated that satellites may effectively broadcastall their downstream traffic to all receivers that are tuned to aparticular carrier, beam, etc. As such, unicasting or multicasting toone or more user systems 110 may, in fact, involve broadcasting the dataover the satellite link and also broadcasting control data to directreceivers to either accept or ignore relevant portions of the broadcastdata. Notably, while some system resources may be expended in setting upa multicast service flow 515 and in related logistics, it “costs” thesatellite communications system substantially the same bandwidthresources to send a packet to one user system 110 or to all user systems110 (e.g., on a particular spot beam).

Similarly, in another embodiment, the client-server communication link125 includes a cable communications link. For example, a cable companymay run a cable line to a neighborhood aggregator, from which individualcoaxial lines communicate last mile traffic to individual households.Each individual coaxial cable may carry all the traffic for the entireneighborhood, even where some of that traffic is destined only forparticular households. As in the satellite embodiment described above,since all the cable subscriber households in the same neighborhoodeffectively receive all the traffic, bandwidth resources can be sharedby multicasting traffic, where appropriate. Of course, satellite andcable networks are only two illustrative embodiments of client-servercommunication links 125. Embodiments of the client-server communicationlink 125 can include any type of communications link that has limitedbandwidth resources, where the bandwidth resources can be at leastpartially shared through multicasting.

It will now be appreciated that embodiments of the client-servercommunication link 125, and the resulting optimizer tunnels 105,effectively provide transparent acceleration functionality to the usersystems 110. This functionality will be described in more detail withrespect to illustrative systems in FIGS. 2-5. FIG. 2 shows a blockdiagram of an embodiment of a satellite communications system 200 havinga server system 220 in communication with multiple user systems 110 viaa satellite 205 over multiple spot beams 235, according to variousembodiments. The server system 220 may include any server components,including base stations 215, gateways 217, etc. A base station 215 issometimes referred to as a hub or ground station. In certainembodiments, as described below, the base station 215 has functionalitythat is the same or different from a gateway 217. For example, asillustrated, a gateway 217 provides an interface between the network 140and the satellite 205 via a number of base stations 215. Variousembodiments provide different types of interfaces between the gateways217 and base stations 215. For example, the gateways 217 and basestations 215 may be in communication over leased high-bandwidth lines(e.g., raw Ethernet), a virtual private large-area network service(VPLS), an Internet protocol virtual private network (IP VPN), or anyother public or private, wired or wireless network. Embodiments of theserver system 220 are in communication with one or more content servers150 via one or more networks 140.

In some embodiments, the gateway 217 is configured to implementrelatively simple routing functions. For example, the gateway 217 mayreceive traffic from the network 140, determine which of the basestations 215 should receive the traffic, and route the trafficaccordingly. In other embodiments, the gateway 217 performs relativelycomplex functions, including, for example, network security, accounting,content acceleration, trend analysis, signal processing and/or encoding,etc. In still other embodiments, the gateway 217 and the base stations215 share some or all of the desired network functionality. For example,it may be desirable to perform certain functions in one location,perform other functions in a distributed manner, and perform still otherfunctions in a redundant manner.

As traffic traverses the satellite communications system 200 in multipledirections, the gateway 217 may be configured to implementmulti-directional communications functionality. For example, the gateway217 may send data to and receive data from the base stations 215.Similarly, the gateway 217 may be configured to receive data andinformation directed to one or more user systems 110, and format thedata and information for delivery to the respective destination devicevia the satellite 205; or receive signals from the satellite 205 (e.g.,from one or more user systems 110) directed to a destination in thenetwork 140, and process the received signals for transmission throughthe network 140.

In one embodiment, the satellite communications system 200 includes anumber of gateways 217 distributed over a large geographic region. Eachgateway 217 is in communication with the network 140 via a high-speedconnection (e.g., a dedicated high-bandwidth fiber link). Each gateway217 is also in communication with, and handles communications for, up totwenty base stations 215 (e.g., twenty feeder links). Each of the twentybase stations 215 is configured to service up to four user links bycommunicating content for those user links to the satellite 205 using anantenna 210.

In various embodiments, one or more of the satellite links are capableof communicating using one or more communication schemes. In variousembodiments, the communication schemes may be the same or different fordifferent links. The communication schemes may include different typesof coding and modulation combinations. For example, various satellitelinks may communicate using physical layer transmission modulation andcoding techniques using adaptive coding and modulation schemes, etc. Thecommunication schemes may also use one or more different types ofmultiplexing schemes, including Multi-Frequency Time-Division MultipleAccess (“MF-TDMA”), Time-Division Multiple Access (“TDMA”), FrequencyDivision Multiple Access (“FDMA”), Orthogonal Frequency DivisionMultiple Access (“OFDMA”), Code Division Multiple Access (“CDMA”), orany number of other schemes.

Embodiments of the satellite 205 may be implemented as a geostationarysatellite 205, a low earth orbit (“LEO”) satellite 205, or aerialpayloads not in orbit and held aloft by planes, blimps, weatherballoons, etc. Other embodiments could have a number of satellites 205instead of just one. In one embodiment, the satellite 205 is configuredas a “bent pipe” satellite, wherein the satellite 205 may frequencyconvert the received carrier signals before retransmitting these signalsto their destination, but otherwise perform little or no otherprocessing on the contents of the signals. There could be a singlecarrier signal for each service spot beam 235 or multiple carriers indifferent embodiments. Similarly, single or multiple carrier signalscould be used for feeder spot beams. A variety of physical layertransmission modulation and coding techniques may be used by thesatellite 205 in accordance with certain embodiments, including thosedefined with the DVB-S2 standard. For other embodiments, a number ofconfigurations are possible (e.g., using LEO satellites, mesh networks,star networks, etc.).

The satellite 205 may operate in a multi-beam mode, transmitting anumber of spot beams 235, each directed at a different region of theearth. Each spot beam 235 may be associated with one of the user links,and used to communicate between the satellite 205 and a large group(e.g., thousands) of user systems 110 (e.g., user terminals 230 withinthe user systems 110). The signals transmitted from the satellite 205may be received by one or more user systems 110, via a respective userantenna 225. In some embodiments, some or all of the user systems 110include one or more user terminals 230 and one or more CPE devices 260.User terminals 230 may include modems, satellite modems, routers, or anyother useful components for handling the user-side communications.Reference to “users” should be construed generally to include any user(e.g., subscriber, consumer, customer, etc.) of services provided overthe satellite communications system 200 (e.g., by or through the serversystem 220).

In a given spot beam 235, some or all of the users (e.g., user systems110) serviced by the spot beam 235 may be capable of receiving all thecontent traversing the spot beam 235 by virtue of the fact that thesatellite communications system 200 employs wireless communications viavarious antennae (e.g., 210 and 225). However, some of the content maynot be intended for receipt by certain customers. As such, the satellitecommunications system 200 may use various techniques to “direct” contentto a user or group of users. For example, the content may be tagged(e.g., using packet header information according to a transmissionprotocol) with a certain destination identifier (e.g., an IP address),use different modcode points that can be reliably received only bycertain user terminals 230, send control information to user systems 110to direct the user systems 110 to ignore or accept certaincommunications, etc. Each user system 110 may then be adapted to handlethe received data accordingly. For example, content destined for aparticular user system 110 may be passed on to its respective CPE 260,while content not destined for the user system 110 may be ignored. Insome cases, the user system 110 stores information not destined for theassociated CPE 260 for use if the information is later found to beuseful in avoiding traffic over the satellite link, as described in moredetail below.

In some embodiments, each user system 110 implements a client optimizer120 that is in communication with a server optimizer 130 located in theserver system 220 (e.g., in the gateway 217). The client optimizers 120and server optimizer 130 may act to create a virtual tunnel between theuser systems 110 and the content servers 150, as described withreference to FIG. 1A. In a topology, like the satellite communicationssystem 200 shown in FIG. 2, vast amounts of traffic may traverse variousportions of the satellite communications system 200 at any given time.As discussed above, at least some of the traffic traversing the networkmay be intercepted by the server optimizer 130 for further processingand for additional functionality. The functionality of the serveroptimizer 130 may also be assisted and/or exploited by other componentsof the server system 220 and the user systems 110. Some of this andother functionality of components of an illustrative server system 220and an illustrative user system 110 are described with reference tovarious types of functional blocks in FIGS. 3 and 4, respectively.

FIG. 3 shows a simplified block diagram 300 illustrating an embodimentof a server system 220 coupled between a network 140 and an antenna 210,according to various embodiments. The server system 220 has a number ofcomponents, including a network interface module 310, a modemtermination module 330, and a server-side transceiver module 360.Components of the server system 220 may be implemented, in whole or inpart, in hardware. Thus, they may include one or more ApplicationSpecific Integrated Circuits (ASICs) adapted to perform a subset of theapplicable functions in hardware. Alternatively, the functions may beperformed by one or more other processing units (or cores), on one ormore integrated circuits (ICs). In other embodiments, other types ofintegrated circuits may be used (e.g., Structured/Platform ASICs, FieldProgrammable Gate Arrays (FPGAs), and other Semi-Custom ICs), which maybe programmed. Each may also be implemented, in whole or in part, withinstructions embodied in a computer-readable medium, formatted to beexecuted by one or more general or application specific controllers.

Embodiments of the server system 220 receive data from the network 140(e.g., the network 140 of FIG. 1A), including data originating from oneor more content servers 150 (e.g., or other types of servers, asdiscussed above) and destined for one or more users in a spot beam(e.g., at a user system 110 in a spot beam 235, as shown in FIG. 2). Thedata is received at the network interface module 310, which includes oneor more components for interfacing with the network 140. For example,the network interface module 310 includes a network switch and a router.

In some embodiments, the network interface module 310 interfaces withother modules, including a third-party edge server 312 and/or a trafficshaper module 314. The third-party edge server 312 may be adapted tomirror content (e.g., implementing transparent mirroring, like would beperformed in a point of presence (“POP”) of a content delivery network(“CDN”)) to the server system 220. For example, the third-party edgeserver 312 may facilitate contractual relationships between contentproviders and service providers to move content closer to users in acommunications network (e.g., the satellite communications network 200of FIG. 2). The traffic shaper module 314 controls traffic from thenetwork 140 through the server system 220, for example, to help optimizeperformance of the communications system (e.g., by reducing latency,increasing effective bandwidth, etc.). In one embodiment, the trafficshaper module 314 delays packets in a traffic stream to conform to apredetermined traffic profile.

Traffic is passed from the network interface module 310 to one or moreprocessing modules. In some embodiments, the processing modules includea server-side accelerator module 350, a scheduler module 335, andsupport modules 346. In some embodiments, all traffic from the networkinterface module 310 is passed to the server-side accelerator module 350for handling, as described more fully below. In other embodiments, someor all of the traffic from the server-side accelerator module 350 ispassed to the support modules 346. For example, in one embodiment,real-time types of data (e.g., User Datagram Protocol (“UDP”) datatraffic, like Internet-protocol television (“IPTV”) programming) bypassthe server-side accelerator module 350, while non-real-time types ofdata (e.g., Transmission Control Protocol (“TCP”) data traffic, like webvideo) are routed through the server-side accelerator module 350 forprocessing. Embodiments of the server-side accelerator module 350provide various types of application, WAN/LAN, and/or other accelerationfunctionality. In one embodiment, the server-side accelerator module 350implements functionality of AcceleNet applications from IntelligentCompression Technologies, Inc. (“ICT”), a division of ViaSat, Inc. Thisfunctionality may be used to exploit information from application layersof the protocol stack (e.g., layers 4-7 of the IP stack) through use ofsoftware or firmware operating in the user system 110 (e.g., in the userterminal 230 and/or the CPE 260).

In some embodiments, the server-side accelerator module 350 is adaptedto provide high payload compression. This allows faster transfer of thedata and enhances the effective capacity of the network. The server-sideaccelerator module 350 can also implement protocol-specific methods toreduce the number of round trips needed to complete a transaction, suchas by prefetching objects embedded in HTTP pages. In other embodiments,functionality of the server-side accelerator module 350 is closelyintegrated with the satellite link through other modules, including thesupport modules 346, the scheduler module 335, the modem terminationmodule 330, etc., to reduce upload bandwidth requirements and/or to moreefficiently schedule to the satellite link. For example, the link layermay be used to determine whether packets are successfully delivered, andthose packets can be tied more closely with the content they supportedthrough application layer information. In certain embodiments, theseand/or other functions of the server-side accelerator module 350 areprovided by a server optimizer 130 resident on (e.g., or incommunication with) the server-side accelerator module 350.

In some embodiments, the server optimizer 130 is implemented withmultiple servers. Each of the multiple servers may be configured tohandle a portion of the traffic passing through the server-sideaccelerator module 350. It is worth noting that functionality of variousembodiments described herein use data which, at times, may be processedacross multiple servers. As such, one or more server management modulesmay be provided for processing (e.g., tracking, routing, partitioning,etc.) data across the multiple servers. For example, when one serverwithin the server optimizer 130 receives a request from a user (e.g.,from a user system 110 on a spot beam 235, as shown in FIG. 2), theserver management module may process that request in the context ofother requests received at other servers in the server optimizer 130. Inone embodiment, coordination between servers is implemented in supportof singular storage of data. For example, it may be desirable to avoidcaching the same byte sequence twice in two servers that are incommunication with each other (e.g., where both servers are part of astorage area network 322 (“SAN”) in the server system 220). In anotherembodiment, servers are configured to communicate to facilitate theidentification of deltacasting (e.g., including multicasting and/orother) opportunities, as described more fully below.

It will be appreciated that, while the server optimizer 130 isillustrated as part of the server system 220, this should not beconstrued as limiting the location or implementation of the serveroptimizer 130. In one embodiment, the server optimizer 130 isimplemented by a server in communication with the server system 220 overthe network 140. For example, a third party may lease server space thatis accessible over the Internet or a private connection (e.g., ahigh-speed fiber connection). The leased server space may be used forserving the server optimizer 130.

Data processed by the server-side accelerator module 350 may passthrough the support modules 346 to the scheduler module 335. Embodimentsof the support modules 346 include one or more types of modules forsupporting the functionality of the modem termination module 330, forexample, including a multicaster module 340, a fair access policy(“FAP”) module 342, and an adaptive coding and modulation (“ACM”) module344. In certain embodiments, some or all of the support modules 346include off-the-shelf types of components.

Embodiments of the multicaster module 340 provide various functionsrelating to multicasting of data over the links of the communicationssystem. Certain embodiments of the multicaster module 340 use datagenerated by other processing modules (e.g., the server-side acceleratormodule 350) to prepare traffic for multicasting. For example, themulticaster module 340 may prepare datagrams as a multicast stream.Other embodiments of the multicaster module 340 perform more complexmulticasting-related functionality. For example, the multicaster module340 may contribute to determinations of whether data is unicast ormulticast to one or more users (e.g., using information generated by theserver-side accelerator module 350), what modcodes to use, whether datashould or should not be sent as a function of data stored at destinationuser terminals 230, how to handle certain types of encryption, etc.

Embodiments of the accounting module 342 implement variousaccounting-related functions. In one embodiment, the accounting module342 collects data from multiple components to determine how much networkusage to attribute to a particular user. For example, the accountingmodule 342 may determine how to count upload or download traffic againsta user's fair access policy (FAP). In another embodiment, the accountingmodule 342 dynamically adjusts FAPs according to various network linkand/or usage conditions. For example, the accounting module 342 mayadjust FAPs to encourage network usage during lower traffic times. Inyet another embodiment, the accounting module 342 affects the operationof other components of the modem termination module 330 as a function ofcertain FAP and/or other accounting conditions. For example, theaccounting module 342 may direct the multicaster module 340 to multicastcertain types of data or to prevent certain users from joining certainmulticast streams as a function of FAP or other considerations.

Embodiments of the ACM module 344 implement various ACM functions. Forexample, the ACM module 344 may track link conditions for certain spotbeams, users, etc., for use in dynamically adjusting modulation and/orcoding schemes. In some embodiments, the ACM module 344 may helpdetermine which users should be included in which customer groupings ormulticast streams as a function of optimizing resources through modcodesettings. In certain embodiments, the ACM module 344 implementsACM-aware encoding of data adapted for progressive encoding. Forexample, MPEG-4 video data may be adapted for progressive encoding inlayers (e.g., a base layer and enhancement layers). The ACM module 344may be configured to set an appropriate modcode separately for eachlayer to optimize video delivery.

When traffic has been processed by the server-side accelerator module350 and/or the support modules 346, the traffic is passed to thescheduler module 335. Embodiments of the scheduler module 335 areconfigured to provide various functions relating to scheduling the linksof the communications system handled by the server system 220. Forexample, the scheduler module 335 may manage link bandwidth byscheduling license grants within a spot beam.

In some embodiments, functionality of the server system 220 involvescommunication and interaction with the SAN 322. Embodiments of the SAN322 include a shared storage module 320, which may include any usefultype of memory store for various types of functionality of the serversystem 220. For example, the shared storage module 320 may includevolatile or non-volatile storage, servers, files, queues, etc. Incertain embodiments, the SAN 322 further includes a captive edge server325, which may be in communication with the shared storage module 320.In some embodiments, the captive edge server 325 provides functionalitysimilar to that of the third-party edge server 312, including contentmirroring. For example, the captive edge server 325 may facilitatedifferent contractual relationships from those of the third-party edgeserver 312 (e.g., between the server system 220 provider and variouscontent providers). In certain embodiments, the captive edge server 325and/or the third-party edge server 312 are in communication withserver-side storage (e.g., within the SAN 322).

It will be appreciated that components of the server system 220 mayprovide many different types of functionality. For example, someembodiments oversee a variety of decoding, interleaving, decryption, andunscrambling techniques. Other embodiments manage functions applicableto the communication of content downstream through a satellite (e.g.,the satellite 205 of FIG. 2) to one or more users (e.g., user systems110 of FIG. 2). As described more fully below with reference to variousembodiments, the server system 220 may handle different types of trafficin different ways. For example, some uses of the communications systeminvolve contractual relationships and/or obligations with third-partycontent providers to interface with their edge servers (e.g., throughthe third-party edge server 312), while other uses involve locally“re-hosting” certain content (e.g., through the captive edge server325). Further, some use cases handle real-time types of data (e.g., UDPdata) differently from non-real-time types of data (e.g., TCP data).Many other uses are possible.

In certain embodiments, some or all of these downstream communicationsfunctions are handled by the server-side transceiver module 360.Embodiments of the server-side transceiver module 360 encode and/ormodulate data, using one or more error correction techniques, adaptiveencoding techniques, baseband encapsulation, frame creation, etc. (e.g.,using various modcodes, lookup tables, etc.). Other functions may alsobe performed by the server-side transceiver module 360 or othercomponents of the server system 220, including upconverting, amplifying,filtering, tuning, tracking, etc. For example, in the context of thesatellite communications system 200 of FIG. 2, the server-sidetransceiver module 360 may communicate data to one or more antennae 210for transmission via the satellite 205 to the user systems 110.Embodiments of the server system 220 also include the modem terminationmodule 330 for receiving modem traffic over the satellite link fromusers. In some embodiments, the modem termination module 330 isconfigured substantially as a satellite modem termination system(“SMTS”).

In other embodiments, downstream functions and or other functions of theserver system 220 are centralized and/or distributed according tovarious embodiments of the invention. For example, as shown in FIG. 2, aserver system 220 may include a number of base stations 215, gateways217, and/or other components (e.g., hubs, cross-connects, cores, etc.).Similarly, in other types of communications systems, multiple serversystem 220 components may perform various functions on the server-sideof the communications system. In some embodiments, substantially eachserver system 220 node (e.g., each base station 215, gateway 217, etc.)is capable of performing substantially all the server system 220functionality. In other embodiments, much of the advanced processingserver system 220 functionality is implemented in edge nodes (e.g., basestations 215) of the server system 220, while other nodes (e.g.,gateways 217, cores, cross-connects, etc.) provide more basic routingand/or switching functions. In still other embodiments, edge nodefunctionality is fairly limited, while advanced processing functions aremore centralized (e.g., in gateways 217, core nodes, etc.).

As described above (e.g., with reference to FIGS. 1 and 2), the serversystem 220 communicates with one or more user systems 110 configured toperform various user-side (e.g., client-side) communications functions.FIG. 4 shows a simplified block diagram of an embodiment of a usersystem 110 a, including an embodiment of a user terminal 230 coupledbetween a user antenna 225 and a CPE 260, according to variousembodiments. Some embodiments of the user system 110 are configured, asshown in FIG. 2, to communicate over a satellite communications system200 by interfacing with a server system 220 over a satellite link (e.g.,the server system 220 of FIG. 3). Interfacing and other functionality ofthe user system 110 may be provided by components of the user terminal230, including a terminal transceiver module 410, data processingmodules 415, and a client storage module 437. Embodiments of the dataprocessing modules 415 include a MAC module 450, a terminal acceleratormodule 430, and a routing module 420.

The components may be implemented, in whole or in part, in hardware.Thus, they may include one or more ASICs adapted to perform a subset ofthe applicable functions in hardware. Alternatively, the functions maybe performed by one or more other processing modules (or cores), on oneor more integrated circuits. In other embodiments, other types ofintegrated circuits may be used (e.g., Structured/Platform ASICs, FPGAs,and other Semi-Custom ICs), which may be programmed. Each may also beimplemented, in whole or in part, with instructions embodied in acomputer-readable medium, formatted to be executed by one or moregeneral or application specific processors.

A signal from the user antenna 225 is received by the user terminal 230at the terminal transceiver module 410. Embodiments of the terminaltransceiver module 410 may amplify the signal, acquire the carrier,and/or downconvert the signal. In some embodiments, this functionalityis performed by other components (either inside or outside the userterminal 230).

In some embodiments, data from the terminal transceiver module 410(e.g., the downconverted signal) is communicated to the data processingmodules 415 for processing. For example, data is communicated to the MACmodule 450. Embodiments of the MAC module 450 prepare data forcommunication to other components of, or in communication with, the userterminal 230, including the terminal accelerator module 430, the routingmodule 420, and/or the CPE 260. For example, the MAC module 450 maymodulate, encode, filter, decrypt, and/or otherwise process the data tobe compatible with the CPE 260.

In some embodiments, the MAC module 450 includes a pre-processing module452. The pre-processing module 452 implements certain functionality foroptimizing the other components of the data processing modules 415. Insome embodiments, the pre-processing module 452 processes the signalreceived from the terminal transceiver module 410 by interpreting (e.g.,and decoding) modulation and/or coding schemes, interpreting multiplexeddata streams, filtering the digitized signal, parsing the digitizedsignal into various types of information (e.g., by extracting thephysical layer header), etc. In other embodiments, the pre-processingmodule 452 pre-filters traffic to determine which data to route directlyto the routing module 420, and which data to route through the terminalaccelerator module 430 for further processing.

Embodiments of the terminal accelerator module 430 provide substantiallythe same functionality as the server-side accelerator module 350,including various types of applications, WAN/LAN, and/or otheracceleration functionality. In one embodiment, the terminal acceleratormodule 430 implements functionality of AcceleNet™ applications, likeinterpreting data communicated by the server system 220 using highpayload compression, handling various prefetching functions, parsingscripts to interpret requests, etc. In certain embodiments, these and/orother functions of the terminal accelerator module 430 are provided by aclient optimizer 120 resident on (e.g., or in communication with) theterminal accelerator module 430. Notably, in some embodiments, theclient optimizer 120 is implemented as client optimizer 120 a on theuser terminal 230 and/or client optimizer 120 b on the CPE 260 b. Datafrom the MAC module 450 and/or the terminal accelerator module 430 maythen be routed to one or more CPEs 260 by the routing module 420.

In some embodiments, output from the data processing modules 415 and/orthe terminal accelerator module 430 is stored in the client storagemodule 437 a. Further, the data processing modules 415 and/or theterminal accelerator module 430 may be configured to determine what datashould be stored in the client storage module 437 a and which datashould not (e.g., which data should be passed to the CPE 260). It willbe appreciated that the client storage module 437 a may include anyuseful type of memory store for various types of functionality of theuser system 110. For example, the client storage module 437 a mayinclude volatile or non-volatile storage, servers, files, queues, etc.Embodiments of the client storage module 437 a are configured to storesome or all of a client dictionary 435, as described more fully below.

In certain embodiments, storage functionality and/or capacity is sharedbetween an integrated (e.g., on-board) client storage module 437 a andan extended (e.g., off-board) storage module 439 a. For example, theextended storage module 439 a may be implemented in various ways,including as an attached peripheral device (e.g., a thumb drive, USBhard drive, etc.), a wireless peripheral device (e.g., a wireless harddrive), a networked peripheral device (e.g., a networked server), etc.In some embodiments, the user terminal 230 interfaces with the extendedstorage module 439 a through one or more ports 438 a. In one embodiment,functionality of the client storage module 437 is implemented as storageintegrated into or in communication with CPE 260 (e.g., as clientstorage module 437 b in CPE 260 b).

Some embodiments of the CPE 260 are standard CPE 260 devices or systemswith no specifically tailored hardware or software (e.g., shown as CPE260 a). Other embodiments of the CPE 260, however, include hardwareand/or software modules adapted to optimize or enhance integration ofthe CPE 260 with the user terminal 230 (e.g., shown as alternate CPE 260b). For example, the alternate CPE 260 b is shown to include a CPEaccelerator module 462, a CPE processor module 466, and a client storagemodule 437 b. Embodiments of the client storage module 437 b areconfigured to store some or all of the client dictionary 435 b.Embodiments of the CPE accelerator module 462 are configured toimplement the same, similar, or complementary functionality as theterminal accelerator module 430. For example, the CPE accelerator module462 may be a software client version of the terminal accelerator module430. In some embodiments, some or all of the functionality of the dataprocessing modules 415 is implemented by the CPE accelerator module 462and/or the CPE processor module 466. In these embodiments, it may bepossible to reduce the complexity of the user terminal 230 by shiftingfunctionality to the alternate CPE 260 b.

Embodiments of the client storage module 437 b may include any type ofdictionary, object or byte caching, data serving, and/or otherstorage-related components in or in communication with the alternate CPE260 b (e.g., a computer hard drive, a digital video recorder (“DVR”),etc.). In some embodiments, the client storage module 437 b is incommunication with an extended storage module 439 b, for example, viaone or more ports 438 b. Of course, many types of CPE 260 are possible,and the functionality of the CPE 260 may be implemented in a number ofdifferent types of devices or systems. In some embodiments, the CPE 260is a fixed or mobile end device for displaying content to the user, likea television, personal computer, home theater system, cellulartelephone, portable music or video player, personal digital assistant,etc. In other embodiments, the CPE 260 is an intermediate device,configured to communicate to another CPE 260 end device (or even toanother CPE 260 intermediate device). For example, the CPE 260 mayinclude a set-top box, a home networking component (e.g., a router, ahub, a femtocell, etc.), or any other type of intermediate device. Asshown, CPE 260 c is in communication with the user terminal 230indirectly through CPE 260 b, where CPE 260 b is acting as anintermediate device.

Further, in some embodiments, the CPE 260 is integrated, partially orcompletely, with the user terminal 230. For example, a home theatersystem may be built around a main interface component that includes anetwork interface having user terminal 230 functionality, certain CPE260 functionality, and ports for wired or wireless communication withadditional CPE 260 devices. Embodiments of user terminals 230 and/orCPEs 260 may also be configured for compatibility with certaincommunication standards. For example, CPEs 260 may be configured tosupport plug-and-play functionality (e.g., through the Digital LivingNetwork Alliance (DLNA) standard), wireless networking (e.g., throughthe 802.11 standard), etc.

In certain embodiments, the user terminal 230 is configured to transmitdata back to the server system 220. Embodiments of the data processingmodules 415 and the terminal transceiver module 410 are configured toprovide functionality for communicating information back through thecommunications system (e.g., through the satellite communications system200 of FIG. 2 for directing provision of services). For example,information about what is stored in the client dictionary 435 may besent back to the server system 220 for limiting repetitious filetransfers, as described more fully below.

It will be appreciated that the communications system may be used toprovide different types of communication services to users. For example,the satellite communications system 200 of FIG. 2 may provide contentfrom content servers 150, through the network 140, to a user's CPE 260,including Internet content, broadcast television and radio content,on-demand content, voice-over-Internet-protocol (VoIP) content, and/orany other type of desired content. It will be further appreciated thatthis content may be communicated to users in different ways, includingthrough unicast, multicast, broadcast, simulcast, and/or othercommunications.

As described above, a number of additional and/or improvedcommunications functions may be facilitated by exploiting contentsharing and/or other types of opportunities through deltacasting. Forexample, in a typical communication system, like the satellitecommunications system 200 of FIG. 2, multiple customers may request thesame or substantially similar content at the same or different times. Byexploiting this feature of the communication system, it may be possibleto optimize (at least partially) the provision of various communicationservices. For example, link conditions (e.g., bandwidth utilization) maybe improved, enhanced services may be offered to customers, costsrelating to service provision may be reduced, etc.

Content sharing may be implemented in many different ways, according toembodiments. For example, certain content may be multicast to a numberof users in a spot beam, thereby allowing multiple user systems 110 toshare channels (i.e., potentially increasing effective throughput).Rather than transmitting a copy of the content to each requesting userthrough a private unicast channel, fewer copies of the content may beshared by multiple users. In certain embodiments, custom oroff-the-shelf components are used to provide this functionality byevaluating multiple communication streams and collapsing them into asingle stream within some tolerance (e.g., a small “jitter window,”accounting for inter-packet delay variances). In other embodiments,dedicated components in the server system 220 implement thisfunctionality.

According to various embodiments, deltacasting and related functionalitymay be implemented at least partially through client-serverinteractions. As discussed above, a server optimizer 130 may determinewhat content is traversing the various links in the communication systemusing fingerprints. For example, the fingerprints may be used toidentify fingerprint trends (e.g., patterns of byte-sequencecommunications) and/or to identify actual content features (e.g.,information from layers 4-7 of the OSI IP protocol stack). Thesedeterminations may then be used to identify and exploit opportunitiesfor improving the communication services over the communications system.

FIG. 5 shows a block diagram of an embodiment of a communications system500, illustrating client-server interactivity through a client optimizer120 and a server optimizer 130, according to various embodiments. Insome embodiments, the communications system 500 is an embodiment of thecommunications system 100 a of FIG. 1A or the satellite communicationssystem 200 of FIG. 2. As shown, the communications system 500facilitates communications between a user system 110 and one or morecontent servers 150 via at least one client-server communication link125 and at least one content network link 135. For example, interactionsbetween the client optimizer 120 and the server optimizer 130effectively create a tunnel 505 between the user system 110 and thecontent servers 150. In some embodiments, the content network link 135includes links through a network 140, like the Internet. Also, asillustrated, embodiments of the client-server communication link 125support one or more unicast service flows 525 and one or more multicastservice flows 515.

In some embodiments, the user system 110 includes a client graphicaluser interface (GUI) 512, a web browser 514, and a redirector 516. Theclient GUI 512 may allow a user to configure performance aspects of theuser system 110 (e.g., or even aspects of the greater communicationssystem 500 in some cases). For example, the user may adjust compressionparameters and/or algorithms, alter content filters (e.g., for blockingillicit websites), or enable or disable various features used by thecommunications system 500. In one embodiment, some of the features mayinclude network diagnostics, error reporting, as well as controlling,for example, components of the client optimizer 120 and/or the serveroptimizer 130.

In one embodiment, the user selects a universal recourse locator (URL)address through the client GUI 512 which directs the web browser 514(e.g., Internet Explorer®, Firefox®, Netscape Navigator®, etc.) to awebsite (e.g., cnn.com, google.com, yahoo.com, etc.). The web browser514 may then issue a request for the website and associated objects(i.e., a content set) to the Internet. It is worth noting that the webbrowser 514 is shown for illustrative purposes only. While embodimentsof the user system 110 may typically include at least one web browser514, user systems 110 may interact with content providers 150 in anumber of different ways without departing from the scope of theinvention. Also, as discussed above, various types of content sets maybe requested in various ways (e.g., feed links may be requested throughfeed readers, etc.).

The content set request from the user system 110 (e.g., from the webbrowser 514) may be intercepted by the redirector 516. It is worthnoting that embodiments of the redirector 516 are implemented in variousways. For example, embodiments of the redirector 516 are implementedwithin a user modem as part of the modem's internal routingfunctionality. The redirector 516 may send the request to the clientoptimizer 120. It is worth noting that the client optimizer 120 is shownas separate from the user system 110 (e.g., in communication over alocal bus, on a separate computer system connected to the user system110 via a high speed/low latency link, like a branch office LAN subnet,etc.). However, embodiments of the client optimizer 120 are implementedas part of the user system 110 in any useful client-side location,including as part of a user terminal, as part of a user modem, as partof a hub, as a separate hardware component, as a software application onthe client machine, etc.

In one embodiment, the client optimizer 120 includes an object processor522 a. The object processor 522 a may be configured to perform a numberof different processing functions, including Java parsing and protocolprocessing. Embodiments of the object processor 522 a may processhypertext transfer protocol (HTTP), file transfer protocol (FTP),various media protocols, metadata, header information, and/or otherrelevant information from the request data (e.g., packets) to allow theclient optimizer 120 to perform its optimizer functions. For example,the request may be processed by the object processor 522 a to determinewhich objects are being requested and whether data needed to generatethe requested object is already stored locally (e.g., from a prefetchoperation, a pre-positioning operation, a multicast caching operation, aprevious deltacasting operation, etc.). Various embodiments exploit bytesequences stored locally in the client dictionary 435. Certainembodiments may also exploit object data and/or other data storedlocally (e.g., in browser cache, etc.), where appropriate.

In some embodiments, the object processor 522 a sends the processedrequest data to a deltacast coder 524 a. The deltacast coder 524 a mayencode the request into a compressed version of the request using one ormore data compression algorithms. For example, these algorithms mayemploy dictionary coding with the client dictionary 435 configured tostore strings so that data from previous web objects can be used tocompress data from new pages. Of course, other types of coding arepossible according to other embodiments of the deltacast coder 524 a.

The processed and/or coded request data may then be further processed bya unicast processor 528 a in some embodiments in preparation forcommunicating the data over the client-server communication link 125(e.g., as private IP traffic). In various embodiments, the unicastprocessor 528 a processes the data according to one or more protocols,for example a unicast protocol, depending at least on the type ofcommunication links implemented as part of the client-servercommunication link 125. For example, the client-server communicationlink 125 may include a wireless link, a cellular link, a satellite link,a dial-up link, etc. In certain embodiments, the unicast processor 528 ais configured to implement Intelligent Compression Technology's (ICT)transport protocol (ITP). In one embodiment, ITP maintains a persistentconnection between the client optimizer 120 and the server optimizer130. The persistent connection may enable the communications system 500to reduce or eliminate inefficiencies and overhead costs associated withcreating a new connection for each request.

In some embodiments, the communication is received at the other end ofthe client-server communication link 125 by a unicast processor 528 b inthe server optimizer 130. In some embodiments, the unicast processor 528b in the server optimizer 130 is implemented as substantially anidentical component to the unicast processor 528 a in the clientoptimizer 120. In other embodiments, implementions of the unicastprocessors 528 may be tailored to their location (e.g., in the clientoptimizer 120 or the server optimizer 130). When the request data isreceived by the unicast processor 528 b, the unicast processor 528 b mayprocess the request (e.g., as part of a content set, as discussed below)according to the applied one or more protocols. For example, the unicastprocessor 528 b may be configured to implement ITP, such that data sentfrom the unicast processor 528 a according to the ITP protocol can beprocessed accordingly.

As discussed above, the data received at the server optimizer 130 fromthe client optimizer 120 may be coded (e.g., dictionary coded) and/orotherwise processed (e.g., according to one or more protocols, likeHTTP). Embodiments of the server optimizer 130 include an objectprocessor 522 b and a deltacast coder 524 b. In some embodiments, theobject processor 522 b and the deltacast coder 524 b are configured tohandle processing and/or coding of the request data implemented by theobject processor 522 a and the deltacast coder 524 a of the clientoptimizer 120, respectively. For example, embodiments of the objectprocessor 522 b use features of the deltacast coder 524 b and/ordictionary types of information, which may be stored, or modeled, in amodeler module 532 to decode the request data. The request may thus beprocessed (e.g., translated, decoded, etc.) into a format that isaccessible to a source of the requested content (e.g., a website). Ofcourse, in certain embodiments, additional features of the request maybe processed by these or other components. For example, if the requestincludes instructions (e.g., set-level metadata, a cookie, or adirective, such as a “referred by” or type of encoding accepted, etc.),information about the instructions may be stored as part of a model inthe modeler module 532 or another location.

Embodiments of the object processor 522 b may then forward the decodedrequest to an appropriate destination (e.g., a content server 150) overthe content network link 135 (e.g., via a network 140). The contentnetwork link 135 may include, for example, a cable modem connection, adigital subscriber line (DSL) connection, a T1 connection, a fiber opticconnection, etc. As discussed above, in some embodiments of thecommunications system 500, the content network link 135 manifestssubstantially lower latency than that of the client-server communicationlink 125.

Response data may be received by the object processor 522 b, in responseto the request, from the appropriate destination (e.g., the contentserver 150) over the content network link 135. It will be appreciatedthat the response data may include various types of information, such asone or more attachments (e.g., media files, text files, etc.),references to “in-line” objects needed to render a web page, etc. Forexample, as discussed above, the response data may be identified as partof a content set and handled as such. Embodiments of the objectprocessor 522 b may be configured to interpret the response data, whichmay, for example, be received as HTML, XML, CSS, Java Scripts, or othertypes of data. As described more fully below, a fingerprint of theresponse data may be generated by the deltacast coder 524 b (e.g., usingdictionary coding techniques) and used for various types of deltacastingand/or other optimization functions.

The fingerprint may be used to determine how to further handle theresponse data, as described below. In some embodiments, processed and/orcoded (e.g., compressed) response data is sent over the client-servercommunication link 125 to the client optimizer 120. The data may be sentas a unicast service flow 525 from the unicast processor 528 b in theserver optimizer 130 to the unicast processor 528 a in the clientoptimizer 120; and/or the data may be sent as one or more multicastservice flows 515 from the multicast processor 530 b in the serveroptimizer 130 to the multicast processor 530 a in the client optimizer120. In certain embodiments, standard protocols are adapted for use withthe unicast service flows 525 and/or the multicast service flows 515.For example, the Pragmatic General Multicast (“PGM”) protocol, theNegative-Acknowledgment (“NACK”) Oriented Reliable Multicast (“NORM”),or “RFC 3940,” protocol from the Internet Engineering Task Force(“IETF”), or other protocols may be used to implement multicasting.

Further, when the client-server communication link 125 includes multiplemulticast service flows 515, the multicast service flows 515 may beconfigured in various ways. In various embodiments, for example, themulticast service flows 515 are configured to each communicate at adifferent modcode point, on a different spot beam, and/or on a differentcarrier. This may allow for more efficient communication of traffic togroups of user systems 110 having particular characteristics. Forexample, if certain traffic is determined to be destined for a usersystem 110 capable of communicating at a particular modcode point, thetraffic may be multicast on a multicast service flow 515 that operatesat or near this modcode point for maximum efficiency (e.g., rather thanat the lowest modcode point needed to transmit to all user systems 110in the multicast group). While this may, in certain cases, cause some ofthe user systems 110 in the multicast group to be unable to reliablyreceive all the multicast data, there may still be an overallimprovement in the operation of the communications system 500.

In other embodiments, modcodes may be handled (e.g., selected, adapted,optimized, etc.) for various effects. In one embodiment, as describedabove, the modcode is selected according to link conditions between theserver optimizer 130 and the client optimizer 120 associated with arequesting client, if any (i.e., so that at least the requesting clientcan reliably receive the communication). In another embodiment, themodcode is selected so that at least some threshold group (e.g., number)of clients can reliably receive the communication. In still otherembodiments, the modcode is adapted to changes in link conditionsbetween the server optimizer 130 and one or more client optimizers 120.For example, adaptive coding and modulation techniques may be used. Themodcode may be adapted by estimating or monitoring link conditions fromthe server-side (e.g., estimating signal-to-noise ratios, bandwidth,etc.) or via feedback from the client-side. In one embodiment, theclient optimizer 120 communicates information, like whether packets arereliably received, as feedback to the server optimizer for dynamicallyadjusting the modcode.

The data received at the client optimizer 120 from the server optimizer130 may be coded (e.g., dictionary coded) and/or otherwise processed(e.g., according to one or more protocols, like HTTP). Embodiments ofthe object processor 522 a and the deltacast coder 524 a in the clientoptimizer 120 are configured to handle processing and/or decoding of theresponse data, respectively. For example, embodiments of the objectprocessor 522 a use features of the deltacast coder 524 a, includingfunctionality of the client dictionary 435, to decode the response data.Embodiments of the object processor 522 a may then forward the decodedresponse to the user system 110 (or to other components of the usersystem 110, where the client optimizer 120 is part of the user system110). The response may then be used by components of the user system110. For example, a media object received as part of the response datamay be played back through a media player at the user system 110, usedto render a web page through the client web browser 514, etc.

It will be appreciated that, while the above description focuses onbrowser requests and responses to those requests, embodiments of theinvention function within many other contexts. For example, embodimentsof the communication system 500 are used to provide interactive Internetservices (e.g., access to the world-wide web, email communications, fileserving and sharing, etc.), television services (e.g., satellitebroadcast television, Internet protocol television (IPTV), on-demandprogramming, etc.), voice communications (e.g., telephone services,voice-over-Internet-protocol (VoW) telephony, etc.), networking services(e.g., mesh networking, VPN, VLAN, MPLS, VPLS, etc.), and othercommunication services. As such, the “response” data discussed above isintended only as an illustrative type of data that may be received bythe server optimizer 130 from a content source (e.g., a content server150). For example, the “response” data may actually be pushed,multicast, or otherwise communicated to the user without an explicitrequest from the user.

For illustrative purposes, traffic over the communications system 500may be categorized into private-interest traffic and public-interesttraffic. Private-interest traffic may include any traffic for whichmulticasting the traffic to multiple user systems 110 is deemedinefficient. For example, where the traffic is of interest to only oneuser system 110, or a very small number of user systems 110, it may costmore to set up and process a multicast service flow than to simplyunicast the traffic to each interested user system 110. Notably, a usersystem 110 may act as an intermediate node (e.g., a hub, switch, router,etc.) that forwards information to multiple end users. For example, in aLAN, data may be received at the client-side for all computers in theLAN by a switch, which may then forward the data to appropriate users inthe LAN; traffic that is of interest to only one user system 110 may, infact, be of interest to many users within a LAN serviced by the one usersystem 110. Alternatively, each user in the LAN may be considered aseparate user system 110 running a separate client optimizer 120. Assuch, the relevant determination may be, from the perspective of theserver optimizer 130, how many unicast service flows 525 on theclient-server communication link 125 would be needed to unicast the datato all interested users. In contrast to private-interest traffic,public-interest traffic may include any traffic for which multicastingthe traffic to multiple user systems 110 is deemed more efficient thanunicasting the traffic to each interested user system 110.

Notably, a number of types of traffic may be either private-interesttraffic or public-interest traffic, depending on the context. Oneexample is control traffic, which may be used for various types ofcontrol of the communications system. For example, control traffic maybe used to send control signals to the client optimizer 120 to directthe client optimizer 120 to accept a particular multicast service flow515. In one embodiment, individual control traffic is sent as unicastservice flows 525 to particular client optimizers 120. In anotherembodiment, certain control traffic is sent to groups of clientoptimizers 120 (e.g., to some or all of the user systems 110 serviced bya particular spot beam of a satellite communications system) as one ormore multicast service flows 515.

Another type of traffic that may be either private-interest traffic orpublic-interest traffic is media object data. In one embodiment, a firstuser takes video with a digital camera as part of a videoconference witha second user. The video file may be considered private-interesttraffic, as it may be of interest only to the recipient and may never berequested, or even be made accessible, to other users on thecommunications system 500. In another embodiment, a reporter for CNNtakes video with a digital camera as part of a live feed to CNN.com. Thevideo file may be considered public-interest traffic, as it may beaccessed by thousands of users on the communications system 500.

Of course, the determination of whether to classify traffic asprivate-interest traffic or public-interest traffic can be made in anumber of ways and may involve many factors. The factors used to makethe determination may be derived from the traffic itself or from othersources (e.g., from an evaluation of current link conditions or currentsystem usage, from third-party information, etc.). When analyzing thetraffic itself, information may be derived from the header portionand/or the content portion of the datagrams. As noted above, the headerportion may provide straightforward sources of information about thecommunication and/or the content of the communication (e.g., throughprotocol information, metadata, public or proprietary tags, etc.).However, the information from the header portion may often be limitedfrom the perspective of a man-in-the-middle type of server optimizer130. For example, relevant header information may be encoded in aproprietary format, may be misleading as to the underlying by sequence,etc.

The content portion of the traffic received at the server optimizer 130includes the actual objects (e.g. content file data) being sent to usersvia respective user systems 110. It will be appreciated that it may bedifficult or impossible to obtain certain types of information lookingonly at the content portion of the traffic datagrams. Of course, varioustypes of data processing (e.g., statistical analysis) can be used toderive information from the byte sequence, but it may be difficult toderive high-level information, such as the file type associated with thedata. For example, a movie is streamed from a VOD server (e.g., as thecontent server 150) to a user terminal 110. Proprietary tags in theheader portion of the traffic may indicate the name of the movie and thefile type for processing at the user's playback device, while thecontent portion may include only the sequence of bytes that define theactual movie content. When the streaming traffic is intercepted by theserver optimizer 130, the server optimizer 130 may be unable to read theheader portion of the traffic, and may, therefore, be unable to use thatinformation for making multicast and/or other determinations.

Notably, however, there are many cases when requests and/or responsescan be characterized and handled at a set level, for example, accordingto a Content Set identifier and set-level metadata. Set-level handlingmay allow embodiments to perform certain high-level functions withcontent traffic, even while acting as a substantially transparentoptimizer. For example, certain traffic optimizations (e.g.,pre-positioning determinations) may be implemented according only tobyte-level content portion data, while allowing certain high-levelfunctionality to be implemented according to set-levelcharacterizations.

FIG. 6 is a flow diagram of an illustrative method 600 for usingdeltacasting to handle content set traffic over a communications system,according to various embodiments. For the sake of clarity, the method600 is described in the context of the communications system 500 of FIG.5. It will be appreciated, however, that various modifications may bemade to the communications system 500 without limiting the scope of themethod 600.

Embodiments of the method 600 begin at block 602 by receiving a block ofcontent data. For example, the content data block (e.g., file data,streaming data, web object data, etc.) may be received as part oftraffic intercepted by the server optimizer 130 from a content server150 over the content network link 135. In some embodiments, the contentdata block is identified as part of a content set. Where the contentdata block is part of a content data set, the method 600 may get anappropriate content set identifier (“Content Set ID”) in block 604. TheContent Set ID may then be assigned to the block of data in block 606.Of course, there may be many ways to identify data as part of a contentset and to get and assign an appropriate Content Set ID, according tovarious embodiments.

In one set of embodiments, pre-positioning routines are initiated (e.g.,each night at 2:00 am, when a certain level of under-subscription orexcess capacity of a communication link is detected, etc.), whereby oneor more session streams may be established to communicate particularcontent sets. Each session stream may carry a Content Set ID, and allcontent set traffic being communicated over the session stream may betagged with the Content Set ID. For example, in one embodiment, a listof feeds (e.g., RSS feeds) are identified as having popular content, andupdates to these feeds are pushed to all interested clients (e.g., usersthat have subscribed to the feeds) overnight. Each night, a number ofsession streams are created to push the feed updates, and each feedupdate carries a Content Set ID (e.g., generated and/or provided as partof the pre-positioning routine). All file data received as part of eachsession stream may be assumed to relate to the requested content set,and all the file data received on the session stream can be associatedwith the session stream's Content Set ID.

In another set of embodiments, set-level handling is invoked as a resultof a client-side request. A user may request a content set by sending arequest to a content server 150. For example, the user may request a URLthrough the web browser 514, request a feed item through a feed reader,etc. The request may be intercepted by the client optimizer 120 andidentified (e.g., tagged) as a content set request. When a sessionstream is established to handle the request (e.g., to communicateresponse data to the user in response to the user's request), thesession stream may be configured to assign a particular Content Set IDto all its response data traffic received at block 602.

In various embodiments, the Content Set ID may be generated and/orassigned to the session stream at either the client side or the serverside. For example, the server optimizer 130 may be configured to assignthe Content Set ID according to a directive from the client optimizer120 that carries the Content Set ID. Alternatively, the server optimizer130 may determine that the request is for a content set and assign aContent Set ID, accordingly.

In yet another set of embodiments, set-level handling is invoked as aresult of server-side determinations. For example, the server optimizer130 may maintain a list of popular content sets. When a request for thecontent set is received at the server optimizer 130, a session streammay be established and an appropriate Content Set ID may be assigned.For example, a request is received for the homepage of a popular newswebsite that is constantly updated throughout the day, such that theresponse data will include the most up-to-date version of the homepageand its associated content objects (the content set). The serveroptimizer 130 may establish a session stream and associate all theresponse traffic on the session stream with a Content Set ID (e.g., andadditional metadata) that characterizes the data as part of the contentset.

In some embodiments, the server optimizer 130 determines that some orall of a content set is being requested according to fingerprints of thedata generated at block 620, as described below. For example, thefingerprint of a received data block may be analyzed by a content setreferrer (e.g., compared against a global content set fingerprint list)to determine that the received data block is part of a content set. Thedata block may accordingly be tagged with an appropriate Content Set ID.

It will be appreciated that other ways of handling Content Set IDs arepossible, according to other embodiments. Further, embodiments alsoassociate set-level (e.g., URL-level, feed item level, feed channellevel, etc.) metadata with the session stream data and/or with theContent Set ID. For example, each data block may be tagged at the serveroptimizer 130 with certain set-level metadata indicating its associationwith the content set for appropriate handling by the client optimizer120. Alternatively, the Content Set ID may be associated with particularset-level metadata, and the client optimizer 120 may handle all datareceived on the associated session stream according to that set-levelmetadata.

As used herein, “set-level metadata” includes any type of set-levelcharacterization information. For example, set-level metadata may beimplemented as HTTP metadata inserted into the header portion of packetsbeing received on a session stream, as entries in an associativedatabase that indexes received data blocks and associates them withcharacterization information, etc. Embodiments can use the set-levelmetadata to perform certain high-level functions on the data and toaffect multicasting and/or other determinations, while otherwisehandling the received file data at the byte level (e.g., without the useof the file metadata), as described below. For example, the content setcan be handled as such, even while treating the received blocks of filedata as substantially meaningless sequences of bytes.

It is worth noting that some content set requests may not invokeset-level handling. Rather, set-level handling may be restricted tocases where it is deemed efficient to handle the request and resultingresponse data as part of a content set. In some embodiments, a list ofpopular content sets (e.g., and associated Content Set IDs) ismaintained at the server-side and/or the client-side of thecommunications system 500. In other embodiments, various types ofmetrics are maintained to determine whether a requested content set ispopular enough to warrant assigning a Content Set ID and handling thedata as such. In still other embodiments, other types of cost-benefitanalyses are performed to determine whether to invoke set-levelhandling. As such, discussions herein of determining that data is partof a content set may be assumed in cases where the data is determined tobe part of a content set for which set-level handling is invoked.

Further, when the response data is not part of a content set (e.g., ornot part of a content set determined to invoke set-level handling), thedata may be handled in various ways, according to various embodiments.In one embodiment, no Content Set ID is received or assigned at blocks604 and 606, respectively. In another embodiment, one or more types ofContent Set ID are assigned to designate the data block as not part of acontent set (e.g., or, more accurately, as part of a category of contentfor which set-level handling is not invoked). In still otherembodiments, the method 600 is invoked only after set-level handling hasbeen invoked. For example, the method 600 begins at block 602 only aftera session stream has been established for a content set, when therequest is made by a special pre-positioning client (e.g., as describedabove), etc.

Even further, before, during, and/or after the Content Set ID isreceived and assigned at blocks 604 and 606, respectively, someembodiments may determine whether the content data block is a multicastcandidate at block 608. The determination in block 608 may be made as afunction of one or more criteria used to define a multicast prefilter612. In one embodiment, the determination is made by the objectprocessor 522 b.

The multicast prefilter 612 may be defined according to any type ofmulticast or similar filtering criteria known in the art. In oneembodiment, the multicast prefilter 612 is based on the file size of thecontent data block. For example, only files larger than a certainminimum size may be considered for multicasting. In another embodiment,information from the header portion of the traffic is used by themulticast prefilter 612. For example, the multicast prefilter 612 may bedefined to make the initial multicast determination in block 608according to source IP address, host URL, destination IP address, filetype, protocol, HTTP metadata, etc. For example, all video files over acertain size coming from YouTube.com may be considered multicastcandidates, while video files being sent as an email attachment to asingle recipient may not be considered multicast candidates.

In some embodiments, data relevant to the multicast prefilter 612 isenhanced through trusted source relationships. For example, contractualrelationships may be formed with content and service providers to allowvisibility by the service providers into the content traversing thenetwork. Embodiments of the trusted source relationships include accessto encryption keys (e.g., including master keys), authorization tore-serve or re-host content (e.g., through a mirroring relationship asdescribed more fully below), etc. In the context of these relationships,the server optimizer 130 may be able to use certain types of proprietarymetadata to make initial multicasting determinations.

Of course, the determination made in block 608 may or may not relate towhether the received content is part of a content set. In one example,the file data may be associated with a Content Set ID in block 606 andthen determined in block 608 to be part of a file that is too small tobe worth multicasting. In another example, for example where substantialprocessing is needed to determine whether received data is part of acontent set, the determination in block 608 may be used to pre-filterreceived data prior to making the content set determination (e.g., priorto receiving and assigning the Content Set ID in blocks 604 and 606,respectively).

When it is determined at block 608 that the content data block is not amulticast candidate, at block 618, the content data block (e.g., or atleast a portion of the content data block) may be unicast, along withany relevant control data, to the appropriate user system(s) 110. Forexample, as described above, the content data block may be processed bythe object processor 522 b and/or the deltacast coder 524 b, and sent asa unicast service flow 525 over the client-server communication link 125via the unicast processors 528. The data may then be received by theclient optimizer 120, processed and/or decoded, and forwarded, asappropriate, to components of the user system(s) 110.

When it is determined at block 608 that the content data block is amulticast candidate (e.g., according to the multicast prefilter 612criteria), the content data block is further processed by the serveroptimizer 130 to determine if any or all of the content data block will,in fact, be sent over one or more multicast service flows 515. At block620, a fingerprint is generated (e.g., a fingerprint is calculated). Insome embodiments, the fingerprint is generated at block 620 by thedeltacast coder 524 b of the server optimizer 130.

In certain embodiments, the fingerprint is generated using cryptographichash functions (e.g., generated by a Message-Digest algorithm 5 (MD5)technique), non-secure hash functions (e.g., generated by a cyclicredundancy check (CRC) technique), or other similar techniques. In otherembodiments, the fingerprint can be generated in any way, such that theresulting fingerprint can be used to indicate that one particular bytesequence (or a portion of the byte sequence) matches another particularbyte sequence (e.g., or a portion of another byte sequence). Embodimentsof dictionary coding (e.g., particularly delta coding) and relatedtechniques are described in more detail in U.S. patent application Ser.No. 12/477,814, entitled “METHODS AND SYSTEMS FOR UTILIZING DELTA CODINGIN ACCELERATION PROXY SERVERS” (026841-002110US), filed on Jun. 3, 2009,which is incorporated herein by reference for any and all purposes.

In some embodiments, the fingerprint is essentially a compressed versionof the byte sequence. In other embodiments, the fingerprint is achecksum, hash, or other technique applied to some or all of the objectdata. For example, in one embodiment, a checksum of the first megabyteof data in the byte sequence is used as a fingerprint. This fingerprintmay then be compared to other fingerprints to find a match. Notably,embodiments may ultimately seek multicast opportunities and/or otheropportunities for optimization of the communications system 500. Assuch, it may be inefficient to generate fingerprints on very smallblocks of data (e.g., at high densities), since it may not be efficientto exploit opportunities where only small blocks are identified asmatches. Further, decreasing the size of blocks may increase the size ofthe dictionary. Some embodiments, therefore, generate fingerprints at aparticular density determined to be efficient according to parameters ofthe communications system 500 or types of data.

It is worth noting that the traffic may include more than just thecontent data block for which a fingerprint is being generated, or thetraffic may include multiple different content data blocks for whichfingerprints are generated. In one example, a media file is received atthe object processor 522 b of the server optimizer 130. The objectprocessor 522 b and/or the deltacast coder 524 b may strip off data(e.g., header information) that is not needed for generating thefingerprint at block 620. In another example, an email is receivedhaving the media file as an attachment. The object processor 522 band/or the deltacast coder 524 b may perform an extra step of strippingoff the email data, in addition to the header and other data, toeffectively isolate the byte sequence for fingerprint generation atblock 620. Of course, as described above, some embodiments may takesteps to maintain the relationship between the byte sequence and thecontent set, even while other types of information are stripped off

In block 624, the fingerprint is matched against other fingerprints ofother content data blocks in the communications system 500. Determiningwhich other content data blocks are “in the communications system 500”may include different types of analyses for different use cases. Forexample, in one embodiment, it is desirable to know whether thefingerprint indicates a matching content data block already stored at aparticular user system 110 (e.g., in the client dictionary 435, etc.).In another embodiment, it is desirable to know whether the fingerprintindicates a matching data block currently being communicated over aunicast service flow 525 or one or more multicast service flows 515. Invarious embodiments, the modeler module 532 in the server optimizer 130is configured to store models that may be useful for making variousdeterminations (e.g., models of client dictionaries 435, models ofserver-side caches or dictionaries, models of past and current streamssent as either unicast service flows 525 or multicast service flows 515,etc.).

It will be appreciated that a number of different types ofdeterminations may be made, depending on which blocks are beingevaluated to find a match, each opening up potential deltacastingopportunities. One such determination is made in some embodiments inblock 628, where the fingerprint of the content data block generated inblock 620 is compared with blocks from the client dictionary model 632to determine whether there is a match. For example, embodiments of theclient dictionary 435 in the client optimizer 120 represent what isstored at a particular client (e.g., at a user system 110), andembodiments of the modeler module 532 at the server optimizer 130 storea model of the each client dictionary 435. If the content data block isdestined for a particular client, the server optimizer 130 may use themodel of the respective client dictionary 435 stored in the modelermodule 532 to look for matches.

If a match is identified, this indicates that the byte sequence (or theportion of the byte sequence) is already stored local to the client(e.g., in the client's client dictionary 435). In that case, at block636, all or relevant portions of the content data block may becompressed using the dictionary model (e.g. dictionary indexes). Atblock 640, the highly compressed version of the content data block maythen be unicast to the client. In some embodiments, the content datablock is compressed by the server-side deltacast coder 524 b andcommunicated as a unicast service flow 525 to the client optimizer 120via the unicast processors 528.

If no match is found at block 628, one or more types of multicastopportunities are evaluated at block 644. Notably, even when no match isfound at block 628, a number of features of the data may be known. Forexample, it may be assumed, according to blocks 604 and 606, that thedata is part of a content set and associated with a Content Set ID. Itmay be further assumed, according to block 628, that the data does notmatch data already stored in one or more client dictionaries 435. Basedon these assumptions, the data may be assumed to be public-interestcontent (e.g., as described above) that has not yet been multicast topotentially interested users. Further, according to block 608, the datamay already have been determined to be a multicast candidate accordingto the multicast pre-filter 612, which may constitute at least aninitial indication that it would be efficient to multicast the data(e.g., barring other information, such as finding of a match at block628).

For these and/or other reasons, some embodiments of the method 600 mayconsider the data to be multicastable without any further analysis. Forexample, based on the above assumptions, it is determined to beefficient enough to consider the data multicastable without evaluatingany further opportunities or information in block 644. Alternatively,embodiments use the evaluations in block 644 to further refine thedetermination of multicastability of the data. In one embodiment, asdescribed above, all data received at block 602 is assigned a ContentSet ID at block 606, even when the Content Set ID sometimes indicatesthat the data should not be handled at the set level. In block 644, itmay be determined that data carrying a Content Set ID that indicates noset-level handling should be handled as unicast data.

Of course, even where it is determined that the data is multicastable,there may still be reasons not to multicast the data or to affect howthe data is multicast. As such, when multicast opportunities areevaluated in block 644, a determination may be made at block 648 as towhether multicast opportunities exist and if they should be exploited.For example, even where a multicast opportunity exists, it may beinefficient to spend the resources to exploit the opportunity (e.g., toset up a multicast service flow 515). In one embodiment, thecommunications system 500 has limited multicast capacity (e.g., alimited number of available multicast service flows). The determinationin block 648 may account for the availability of multicast serviceflows, including, for example, whether it is possible to use up one ofthe available service flows (or whether it is efficient to use the flowfor this traffic at the expense of taking multicast capacity away fromother traffic). In another embodiment, the Content Set ID is evaluatedin block 648 to determine an appropriate multicast group (e.g., a groupof potentially interested users) for the content set. Based on one ormore factors (e.g., modcode point(s) needed for reliable multicasting tothe multicast group), the data may be unicast, communicated overmultiple unicast and/or multicast service flows, etc. to optimizeforward-link capacity sharing.

Further, multicast opportunities may be evaluated and fingerprintgeneration can be tailored in various ways depending on the types ofopportunities being evaluated (e.g., the fingerprint may, itself, be asequence of bytes or part of a more complex system of determining theassociated byte sequence). By way of example, the fingerprints may beused in the context of identifying multicast opportunities with currentservice flows (e.g., to see if content requested by one user iscurrently being unicast or multicast to other users). To facilitate thistype of identification, one embodiment generates maps having keys beingthe various fingerprints identifying the content data block and payloadsthat provide data about transfers underway or other useful information.

In certain embodiments, the maps are kept to a reasonable size to avoidunnecessary processing of data. For example, techniques are used torestrict the cases where the fingerprint is added to the map. In oneembodiment, protocols that are “uninteresting” are excluded. Forexample, fingerprints may be created only for protocols known (e.g.,predetermined) to be interesting, such as HTTP, certain media downloadprotocols, etc. (e.g., as prefiltered in block 608). In anotherembodiment, small objects are excluded, as described above withreference to block 608. For example, if the size of the requested objectis known (or predictable) in advance, it may be used as a filter—if theobject is smaller than some threshold size, the fingerprint is not addedto the map. When the object size is unknown (or not practicallypredictable), embodiments may wait until at least a minimum amount ofdata has been received, then filter out the noise (e.g., very smallobjects). Of course, it may be important to avoid delaying the map entrytoo long, such that it would cause the optimizer to miss a certain matchwith a new download. In some embodiments, when the download is complete,the fingerprint is removed from the map.

If a determination is made at block 648 that either no multicastopportunities exist, or that the multicast opportunities should not beexploited, the content data block data and/or any related control datais unicast at block 652, where appropriate. In some embodiments,unicasting the data at block 652 involves communicating the data as aunicast service flow 525 to the client optimizer 120 via the unicastprocessors 528. If a determination is made at block 648 that a multicastopportunity exists and should be exploited, the content data block maybe multicast to one or more clients at block 656 (e.g., all userssubscribing to a feed, all users deemed to be potentially interested inthe content, all clients sharing the forward link, etc.). In someembodiments, multicasting the data at block 656 involves communicatingthe content block data over one or more multicast service flows 515 tothe client optimizer 120 via the multicast processors 530.

In certain embodiments, the fingerprint generated in block 620, oranother representation of the data (e.g., the byte sequence itself, acompressed version or a portion of the byte sequence, or a differenttypes of fingerprint) is stored at the server-side for later use by thecommunications system 500. For example, storage of relevant informationmay be useful in generating or identifying future multicastopportunities, tracking and/or characterizing network usage,prefetching, etc. Further, embodiments may store the Client Set IDand/or may store the data representation in association with the ClientSet ID.

It will be appreciated that, in some embodiments, multicasting orunicasting data is implemented in different ways. For example, in thesatellite communications system 200 of FIG. 2, some or all of thereceivers (e.g., user systems 110) in a spot beam 235 may inherently becapable of receiving at least a portion of any traffic being sent overthe spot beam 235 by virtue of being tuned to the appropriate carrier,able to receive data at the current modcode point, etc.; effectively,the satellite communications system 200 broadcasts everything over theair. As such, as discussed above with reference to FIG. 1B, unicastingor multicasting to one or more user systems 110 may, in fact, involvebroadcasting the data over the satellite link and also broadcastingcontrol data to direct receivers to either accept or ignore relevantportions of the broadcast data.

In one illustrative embodiment, it is determined that content requestedby one user has a high probability of being accessed by a group ofnon-requesting users sharing a satellite spot beam on the communicationssystem 500 (e.g., because it is part of a popular content set). Thecontent is broadcast over the satellite link with a stream identifierthat designates it as a multicast stream. The stream identifier may bethe same as or different from the Content Set ID. Control data is alsosent directing user systems 110 associated with the interested users to“listen” to the multicast stream (e.g., to accept, rather than ignore,data with that stream identifier as it is received). In effect, thiscreates a multicast group of the interested users. In differentembodiments, the control data may be communicated to the multicast groupeither as respective unicast service flows 525 to each client via theunicast processors 528 or as part of a multicast control channel sentover a multicast service flow 515 via the multicast processors 530. Itwill be appreciated that, for the sake of bandwidth efficiency,embodiments typically send the control data over the multicast controlchannel. For example, all the user systems 110 may be constantlylistening to the multicast control channel to find out (e.g., amongother things) which streams they should accept. Of course, otherimplementations are possible according to various embodiments forunicasting or multicasting the data over various unicast service flows525 and/or multicast service flows 515 to the client optimizer(s) 120.

Once the data is received at the client optimizer 120, it may be storedat the client-side (e.g., blocks of the data may be stored and indexedby the client dictionary 435). In certain embodiments, storage in theclient dictionary 435 ultimately causes a record of the data to bereflected at the server optimizer 130 if a model of the client-sideclient dictionary 435 is updated (e.g., through synchronization of themodeler module 532). When it is determined in block 648 that the datawill be multicast in block 656 (e.g., and/or when the data is determinedto be unicast in block 652), the data may be compressed and/or otherwisecoded before it is sent over the client-server communication link 125.In one embodiment, the data is zip coded prior to being sent over theclient-server communication link 125. When the zipped data is receivedat the client optimizer 120, the data is added to the client dictionary435.

It is worth noting that the use of fingerprinting (e.g., and/or otherdictionary coding techniques) to make multicasting and relateddeterminations may provide a number of features. One feature is thatdeltacasting opportunities may be identified and/or exploited even wherethere is little or no access to certain metadata. For example, asdiscussed above, the server optimizer generates signatures based on bytelevel data and does not require knowledge of “header portion”information (e.g., file types, proprietary tags, protocol tags, etc.) tomake its determinations.

Another feature is that fingerprinting techniques may allow deltacastingopportunities to be identified, even where the content source or other“header portion” (e.g., metadata) information is different. For example,say viewers are watching the same television show at the same time fromdifferent sources (e.g., different television channels are broadcastingthe same content, different websites are mirroring the same content,etc.). Fingerprinting techniques can find matching blocks, as the blockswill match even where the content sources are different. Similarly,deltacasting opportunities may be identified even where cache-busting,anonymizer, spoofing, mirroring, and/or other techniques are used (e.g.,to alter URLs, to implement content data network (CDN) functionality,etc.).

Still another feature is that deltacasting techniques may be usedtransparently to preserve communications from the perspective of endusers and content sources. In particular, an end user and a contentsource may effectively experience the same byte-for-byte communicationswith or without deltacasting. For example, even though requests and/orresponses are intercepted according to deltacasting embodiments, when auser requests data from a content source, the content source mayultimately provide the same bytes to the end user as if there were aunicast link between the end user and the content source.

It is also worth noting that embodiments allow substantially transparentoptimization of communications while preserving certain legal andbusiness relationships, including, copyright, digital rights management,subscription, and/or other obligations. For example, as discussed above,content data is stored in dictionaries effectively as dissociated blocksof data, such that the content can only be recreated from those blocksusing appropriate dictionary references (e.g., indexes). According tovarious embodiments, those dictionary references are unavailable toclients without a new request from the content source.

In one illustrative embodiment, a first user watches a movie through apopular video-on-demand website by logging into the website usingcredentials (e.g., a user name and password) and viewing the moviethrough an embedded player surrounded by banner advertisements. Based onone or more determinations discussed above, the content set for thewebsite is multicast to the first (requesting) user and to a second(non-requesting) user, and is stored in the second user's clientdictionary 435. The second user's client dictionary 435 may now includedata blocks from a movie that includes copyrighted material, from a websession authenticated according to another user's credentials, fromadvertisements that may be cycled and/or tracked, from web objects thatare designated in metadata as “un-cacheable” etc. As discussed above,embodiments of the client dictionary 435 store the data blocks in such away that it may be effectively impossible for the first user to accessthe movie content directly from the client dictionary 435.

Instead, if the second user later requests the movie, the second user'sexperience may be much the same as that of the first user (e.g., andmuch the same as it would have been had the data not been stored in theclient dictionary 435). For example, the second user may still visit thewebsite using a web browser and may still log in with credentials. Ifauthorized, the second user may still request an authorized, licensedcopy of the movie file from the website, which may then be viewed in theembedded player surrounded by banner advertisements. However, as thedata is received in response to the request, deltacasting techniques areused to fingerprint the data and identify the data as already beingstored in the second user's client dictionary 435. The data may then becommunicated to the second user accordingly, for example, by highlycompressing the data according to a model of the client dictionary 435stored at the server side of the communications system 500 (e.g., theclient dictionary model 632).

As such, the use of deltacasting techniques may preserve legal and otherobligations for content transactions. In the above example, the seconduser is unable to access copyright and/or unauthorized material from theclient dictionary 435. Further, forcing the second user to access thecontent as intended by the content provider (e.g., through theprovider's website) may allow the content provider to preserveadvertising, hosting, and/or other relationships. For example, if thecontent provider happens to offer an advertisement that is alreadystored in the client dictionary, the advertisement may still berequested over the content network link 135 (e.g., thereby providing anyassociated advertisement tracking, revenue, etc.) while also beinghighly compressed over the client-server communications link 125.

It will now be appreciated that use of deltacasting techniques toidentify and/or exploit multicasting opportunities provides certainfeatures. However, using only the byte-level information may providelittle or no content awareness. As such, according to variousembodiments, deltacasting techniques are used to handle data blockstraversing the communications stem 500 according to their byte-leveldata (e.g., transparently and/or agnostically), and certain blocks arecharacterized as parts of content sets and handled accordingly toprovide set-level awareness. For example, as described above,embodiments may implement a transparent optimizer that can performset-level handling even with a dictionary of substantially unordereddata blocks and with little or no knowledge of what content those blocksrepresent.

In one illustrative embodiment, a user requests a popular webpage, andthe response includes blocks of data relating to a number of differenttypes of content originating from a number of different content sources.The optimizer components (e.g., the server optimizer 130) treat eachblock effectively as a meaningless sequence of bytes, such thatfingerprints are generated and some or all multicasting determinationsare made with little or no consideration of what the byte sequencerepresents (e.g., its metadata, file type, etc.). For example, theoptimizer components may function without determining whether theresponse data includes five or fifty separate objects; whether theobjects are of certain types; whether the objects were retrieved fromone or more content sources; whether cache busting, anonymizer,spoofing, and/or other techniques were used, whether there is anyauthentication requirement for accessing the data, etc. Even withoutthis knowledge, however, if the popular webpage is determined to be acontent set (e.g., as described above), the byte sequence can be handledas part of its respective content set.

The set-level handling may provide a number of different types offunctionality. One type of functionality relates to identifying andexploiting content popularity and related metrics. Even without anobject-level awareness of the content traversing the communicationssystem 500 (e.g., without an awareness of which content objects arerepresented by the blocks of data stored at the client dictionaries,etc.), popularity and/or other metrics can be evaluated at the setlevel. For example, as discussed above, a list of popular webpages orfeeds may be maintained, various statistics may be gathered, etc. Themetrics may be used for many types of applications, including for webtracking (e.g., for reporting web traffic statistics), finding usagecorrelations between users, satisfying subscriptions (e.g., where usershave subscribed to a particular feed), etc.

Another type of functionality relates to storage management. Forexample, a user may set preferences (e.g., or have preferences set by aservice provider, have preferences dynamically adjusted according tousage or other factors, etc.) as to storage capacity designated forcertain types of use. In one embodiment, a user selects a number offeeds and/or websites to be automatically updated by a pre-positioningclient. Each feed or website may be associated with a bin (e.g., spacewithin the user's client dictionary 435) having a designated capacity.Similarly, a service provider may set certain bins in the clientdictionary 435 for pre-positioning certain types of data that is likelyto be requested by users. For example, the first Megabyte of data forthe top one-hundred movie downloads of the day may be anticipatorilymulticast to all users and stored in a designated bin in theirrespective client dictionaries 435.

Still another type of functionality relates to using set-level awarenessto maintain a high-level awareness of the contents of a clientdictionary 435. In one embodiment, a browser plug-in may be providedthat augments the user's web browsing experience according to high-levelawareness of dictionary contents. For example, when a result of a websearch points to a webpage stored as a content set in the clientdictionary 435, the search result listing may indicate as such (e.g.,the result may be displayed in a different color, highlighted orflagged, listed separately, priced differently, etc.). Similarly, othertypes of content interfaces (e.g., media players, feed readers, applets,etc.) may be configured to provide the user or any other entity with aset-level indication of client dictionary 435 contents.

In another embodiment, the set-level awareness is used to affect arelationship between the user and a service or content provider. Forexample, the user may contract with a service provider according to aFair Access Policy (FAP), which designates resource usage policies forthe user (e.g., the amount of download bandwidth a user is allowed overeach 24-hour period). In one embodiment, users are encouraged to uselocally stored data by providing a set-level indication of the clientdictionary 435 contents and treating requests for the content asFAP-free or FAP-discounted (e.g., less bandwidth is used to satisfy thecontent set request, and the FAP hit for requesting the content isadjusted accordingly).

Yet another type of functionality relates to using set-level awarenessto police content set determinations. In some cases, even where acontent set is identified, it may be handled differently according tohow the content set is sourced by content sources, used by users, etc.In one example, it is determined that each time a popular webpage isrequested, over ninety-percent of the content set associated with thewebsite is changed (e.g., the website shows the most popular moviedownload across the web at the time the request is made, such that thecontent set may significantly change for each subsequent request). Theoptimizer components may be unaware of which part of the content set ischanging (e.g., because of a lack of object-level awareness), but thecontent set-level changes may nonetheless be determined to be sosignificant as to outweigh any efficiencies gained through furtherset-level handling. For example, in this and other cases, policing thecontent set may cause it to be removed from a list of pre-positioningsets, to not be indicated as available in the client dictionary 435, tonot be offered as FAP-free or FAP-discounted, etc.

The above description is intended to provide various embodiments of theinvention, but does not represent an exhaustive list of all embodiments.For example, those of skill in the art will appreciate that variousmodifications are available within the scope of the invention. Further,while the disclosure includes various sections and headings, thesections and headings are not intended to limit the scope of anyembodiment of the invention. Rather, disclosure presented under oneheading may inform disclosure presented under a different heading. Forexample, descriptions of embodiments of method steps for handlingoverlapping content requests may be used to inform embodiments ofmethods for handling anticipatory requests.

Specific details are given in the above description to provide athorough understanding of the embodiments. However, it is understoodthat the embodiments may be practiced without these specific details.For example, well-known processes, algorithms, structures, andtechniques may be shown without unnecessary detail in order to avoidobscuring the embodiments. Implementation of the techniques, blocks,steps, and means described above may be done in various ways. Forexample, these techniques, blocks, steps, and means may be implementedin hardware, software, or a combination thereof. For a hardwareimplementation, the processing units may be implemented within one ormore application specific integrated circuits (ASICs), digital signalprocessors (DSPs), digital signal processing devices (DSPDs),programmable logic devices (PLDs), field-programmable gate arrays(FPGAs), soft core processors, hard core processors, controllers,micro-controllers, microprocessors, other electronic units designed toperform the functions described above, and/or a combination thereof.Software can be used instead of or in addition to hardware to performthe techniques, blocks, steps and means.

Also, it is noted that the embodiments may be described as a processwhich is depicted as a flowchart, a flow diagram, a data flow diagram, astructure diagram, or a block diagram. Although a flowchart may describethe operations as a sequential process, many of the operations can beperformed in parallel or concurrently. In addition, the order of theoperations may be re-arranged. A process is terminated when itsoperations are completed, but could have additional steps not includedin the figure. A process may correspond to a method, a function, aprocedure, a subroutine, a subprogram, etc. When a process correspondsto a function, its termination corresponds to a return of the functionto the calling function or the main function.

Furthermore, embodiments may be implemented by hardware, software,scripting languages, firmware, middleware, microcode, hardwaredescription languages, and/or any combination thereof. When implementedin software, firmware, middleware, scripting language, and/or microcode,the program code or code segments to perform the necessary tasks may bestored in a machine readable medium such as a storage medium. A codesegment or machine-executable instruction may represent a procedure, afunction, a subprogram, a program, a routine, a subroutine, a module, asoftware package, a script, a class, or any combination of instructions,data structures, and/or program statements. A code segment may becoupled to another code segment or a hardware circuit by passing and/orreceiving information, data, arguments, parameters, and/or memorycontents. Information, arguments, parameters, data, etc. may be passed,forwarded, or transmitted via any suitable means including memorysharing, message passing, token passing, network transmission, etc.

For a firmware and/or software implementation, the methodologies may beimplemented with modules (e.g., procedures, functions, and so on) thatperform the functions described herein. Any machine-readable mediumtangibly embodying instructions may be used in implementing themethodologies described herein. For example, software codes may bestored in a memory. Memory may be implemented within the processor orexternal to the processor. As used herein the term “memory” refers toany type of long term, short term, volatile, nonvolatile, or otherstorage medium and is not to be limited to any particular type of memoryor number of memories, or type of media upon which memory is stored.

Moreover, as disclosed herein, the term “storage medium” may representone or more memories for storing data, including read only memory (ROM),random access memory (RAM), magnetic RAM, core memory, magnetic diskstorage mediums, optical storage mediums, flash memory devices and/orother machine readable mediums for storing information. Similarly, termslike “cache” are intended to broadly include any type of storage,including temporary or persistent storage, queues (e.g., FIFO, LIFO,etc.), buffers (e.g., circular, etc.), etc. The term “machine-readablemedium” includes, but is not limited to, portable or fixed storagedevices, optical storage devices, wireless channels, and/or variousother storage mediums capable of storing that contain or carryinstruction(s) and/or data.

Further, certain portions of embodiments (e.g., method steps) aredescribed as being implemented “as a function of” other portions ofembodiments. This and similar phraseologies, as used herein, intendbroadly to include any technique for determining one element partiallyor completely according to another element. For example, a method mayinclude generating a fingerprint from a first request and generating adetermination “as a function of” the fingerprint. In variousembodiments, the determination may be made in any way, so long as theoutcome of the determination generation step is at least partiallydependant on the outcome of the fingerprint generation step.

While the principles of the disclosure have been described above inconnection with specific apparatuses and methods, it is to be clearlyunderstood that this description is made only by way of example and notas limitation on the scope of the disclosure.

1.-15. (canceled)
 16. A method for multicasting over a communicationssystem having a communications path between a server side of thecommunications system and a plurality of clients, the communicationspath comprising a shared forward link over which bandwidth resources areshared during a multicast communication, the method comprising:maintaining at the server side of the communications system a pluralityof content set identifiers each identifying a content set having aplurality of content objects, the content set having a set-levelprofile; intercepting a data block at the server side of thecommunications system, the data block comprising data block metadata;determining, at the server side of the communications system, that thedata block is being communicated in response to a request by arequesting client for a content object that is part of a content setidentified by one of the plurality of content set identifiers;associating the data block with the content set identifier of theidentified content set; determining, at the server side of thecommunications system, whether to communicate the data block to at leastthe requesting client by determining whether the data block is notpresently stored local to the requesting client as a function of thecontent set identifier of the identified content set; determining, whenit is determined to communicate the data block to at least therequesting client, whether to multicast the data block over thecommunications path to at least the plurality of clients based on theset-level profile of the identified content set; and multicasting thedata block over the communications path when it is determined tomulticast the data block, wherein the content set identifier determineswhether a client of the plurality of clients accepts the data blockmulticast over the communications path to store the data block at theclient.
 17. The method of claim 16, wherein the content set comprisesset-level metadata.
 18. The method of claim 17, wherein the determiningwhether to multicast the data block over the communications path to atleast the plurality of clients according to the content set identifieris at least in part based on the set-level profile for the content setassociated with the data block.
 19. The method of claim 16, wherein themulticasting the data block satisfies a plurality of requests ofdifferent ones of the plurality of clients, the plurality of requestscorresponding to different sources of the data block.
 20. The method ofclaim 16, wherein the determining that the data block is beingcommunicated in response to the request by the requesting client for thecontent object that is part of the content set identified by one of theplurality of content set identifiers is based on the data blockmetadata.
 21. The method of claim 16, wherein the determining whether aclient of the plurality of clients accepts the data block multicast overthe communications path to store the data block is according to at leasta portion of the set-level profile.
 22. The method of claim 16, furthercomprising: when it is determined to store the data block in a clientdictionary according to the content set identifier, storing the datablock in the client dictionary in association with the content setidentifier.
 23. The method of claim 16, wherein a plurality of datablocks from different sources are each associated with the identifiedcontent set.
 24. The method of claim 16, wherein using the content setidentifier to determine whether the client of the plurality of clientsaccepts the data block multicast over the communications path comprises:determining, at the server side of the communications system, whether toinclude the client of the plurality of clients in a multicast group,wherein the data block is multicast over the communications pathaccording to the multicast group.
 25. The method of claim 24, furthercomprising: detecting a link condition affecting communications over thecommunications path with at least a portion of the multicast group; andselecting a modcode as a function of the link condition, wherein thedata block is multicast over the communications path further accordingto the modcode.
 26. The method of claim 16, wherein using the contentset identifier to determine whether the client of the plurality ofclients accepts the data block multicast over the communications pathcomprises: determining, at the client of the plurality of clients,whether to locally store the data block according to the content setidentifier.
 27. The method of claim 16, further comprising: using thecontent set identifier to determine whether the client of the pluralityof clients accepts the data block multicast over the communications pathcomprises determining whether the client of the plurality of clientsaccepts the data block according to at least a portion of the set-levelprofile.
 28. The method of claim 16, further comprising: receiving arequest for the content set at the server side of the communicationssystem; downloading a set of content objects in response to receivingthe request for the content set; and associating the set of contentobjects with the content set identifier, wherein the data block isintercepted as part of downloading the set of content objects.
 29. Themethod of claim 28, wherein the set of content objects are downloadedfrom different sources.
 30. The method of claim 16, wherein the contentset identified by the content set identifier is configured to compriseall data blocks downloaded from a predetermined host.
 31. The method ofclaim 16, wherein the content set identified by the content setidentifier dynamically changes along with changes to the content objectscomprised by the content set.
 32. The method of claim 16, wherein thecontent set comprises a content feed.
 33. The method of claim 16,wherein the content set comprises a webpage.
 34. The method of claim 16,wherein the communications path comprises a satellite link.
 35. A serversystem for multicasting over a communications system having acommunications path between a server side of the communications systemand a plurality of clients, the communications path comprising a sharedforward link over which bandwidth resources are shared during amulticast communication, the server system comprising: an optimizermodule, stored in a memory at the server side of the communicationssystem, the optimizer module, when executed, causing a processor to:maintain at the server side of the communications system a plurality ofcontent set identifiers each identifying a content set having aplurality of content objects, the content set having a set-levelprofile; intercept a data block at the server side of the communicationssystem, the data block comprising data block metadata; determine, at theserver side of the communications system, that the data block is beingcommunicated in response to a request by a requesting client for acontent object that is part of a content set identified by one of theplurality of content set identifiers; and associate the data block withthe content set identifier of the identified content set; acommunications module, stored in the memory, communicatively coupledwith the optimizer module, the communications module, when executed,causing the processor to: determine whether to communicate the datablock to at least the requesting client by determining whether the datablock is not presently stored local to the requesting client as afunction of the content set identifier of the identified content set;and when it is determined to communicate the data block to at least therequesting client, determine whether to multicast the data block overthe communications path to at least the plurality of clients based onthe set-level profile of the identified content set; and multicast thedata block over the communications path.
 36. The server system of claim35, further comprising: a content set generator module, stored in thememory, communicatively coupled with the optimizer module, the contentset generator module, when executed, causing the processor to: receive arequest for the content set; download a set of content objects from aplurality of sources in response to receiving the request for thecontent set; and associate the set of content objects with the contentset identifier, wherein the data block is intercepted by the optimizermodule as part of downloading the set of content objects.